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Identification of Important “Party Line” Information Elements 

and the Implications for Situational Awareness 
in the Datalink Environment 


ABSTRACT 

Air/ground digital datalink communications are an integral component of the FAA’s 
Air Traffic Control (ATC) modernization strategy. With the introduction of datalink into 
the ATC system, there is concern over the potential loss of situational awareness by flight 
crews due to the reduction in the "party line" information available to the pilot. "Party line" 
information is gleaned by flight crews overhearing communications between ATC and 
other aircraft In the datalink environment, party line information may not be available due 
to the use of discrete addressing. Information concerning the importance, availability, and 
accuracy of party line elements was explored through an opinion survey of active air earner 
flight crews. The survey identified numerous important party line elements. These 
elements were scripted into a full-mission flight simulation. The flight simulation 
experiment examined the utilization of party line information by studying subject responses 
to the specific information elements. Some party line elements perceived as important were 
effectively utilized by flight crews in the simulated operational environment. However, 
other party line elements stimulated tittle or no increase in situational awareness. The 
ability to assimilate and use party tine information appeared to be dependent on workload, 
time availability, and the tactical/strategic nature of the situations. In addition, the results of 
both the survey and the simulation indicated that the importance of party line information 
appeared to be greatest for operations near or on the airport This indicates that caution 
must be exercised when implementing datalink communications in these high workload, 
tactical sectors. 


This document is based on the thesis of Alan H. Midkiff submitted in partial fulfillment of 
the degree of Master of Science in Aeronautics and Astronautics at the Massachusetts 
Institute of Technology. 
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1 . Introduction 


Every day thousands of general aviation and scheduled air earner operations utilize 
the services of the current Air Traffic Control (ATC) and airway flight service system. 

Much of this airspace is operating near capacity levels and, in many terminal areas, at or 
beyond full capacity. Air/ground communications are currently handled via VHF radio 
voice communications between the aircraft and various ATC facilities. There are significant 
limitations of the voice system as indicated by the high number of Aviation Safety and 
Reporting System (ASRS) submissions identifying breakdowns and saturation in VHF 
voice channels. For example, of the more than 14,000 ASRS reports received in 1985 and 
1986, one fourth involved problems in air/ground information transfer [1]. 

As part of the Federal Aviation Administration’s (FAA) National Airspace System 
modernization plan, digital datalink communications will be introduced as a means of 
air/ground information exchange between aircraft and ATC facilities. Communications via 
datalink offer potential benefits in increased system safety and efficiency. These would be 
achieved by reducing transmission and interpretation errors and by allowing more 
information to be exchanged between aircraft and ground facilities. On the other hand, the 
transfer of ATC communications from voice to datalink gives rise to numerous human 
factors issues including a possible loss of flight crew situational awareness. Specifically, 
the discrete nature of datalink addressing (where an ATC message is directed exclusively to 
a specific aircraft) may result in a loss of indirect or “party line” information (PLI). 
Crewmembers obtain PLI through frequency monitoring by overhearing communications 
between ATC and other aircraft The identification of important PLI elements is necessary 
to form a basis by which compensatory datalink protocols or strategies can be developed. 

This report documents efforts to identify party line information elements currently 
available in the ATC system and to determine their relative importance. Importance of PLI 
was addressed through an opinion survey of active airline pilots which also explored the 
accuracy, and availability of numerous PLI elements. The most important PLI elements 
were further investigated in a flight simulation study where crew responses to specific 
information elements could be tested. 



The results are presented as follows. Chapter 2 outlines the primary motivation for 
this work, including a discussion of digital datalink development and potential PLI-datalink 
tradeoffs. Chapter 3 describes the design and results of the pilot opinion survey on the 
importance, accuracy, and availability of PLI information elements. Chapter 4 details the 
flight simulation study of PLI usage. A summary of the conclusions is contained in 
Chapter 5. 



2. Background 


The development of the air/ground communication system has paralleled the 
commercial development of civil aviation. The ATC system has had to accommodate both 
increases in air traffic and operating restrictions which result from congestion and noise 
considerations. As the system has become more saturated, the need to move to alternate 
forms of communication has increased. Alternate communication forms, such as datahnk, 
must be evaluated for possible adverse effects as well as advantages, since their 
implementation may result in eliminating benefits such as indirect information sources. An 
example of such an indirect source is the Party Line Information (PLI) found in current 
voice communications. 

2.1. Party Line Information (PLI) 

In the current ATC environment, voice communications are made on a common 
VHF frequency where aircraft, tuned to that particular frequency, can monitor all of the 
transmissions. Some of this “party line” information is used by pilots to increase their 
situational awareness with respect to other aircraft and/or environmental conditions. An 
example of this type of information is turbulence and ride reports. Aircraft.along a similar 
route or altitude often convey turbulence information to controllers that other aircraft can 
overhear. Party line information is also useful for assessing sector congestion and 
controller workload. It has other indirect uses such as providing flight crews with the 
frequent psychological assurance that they are in contact with ATC, i.e. their 
communications equipment is still functioning normally. Voice inflection by controllers 
and pilots can also indicate the urgency of an instruction or situation. Controllers will often 
solicit PLI by requesting information from other aircraft such as enroute ride reports, 
airspeed gain or loss on final approach, and braking action reports on landing roll-out. 
Intentional PLI is also utilized in the transoceanic environment where pilots maintain a 
listening watch on dedicated VHF air-to-air frequencies and also on the universal 
emergency (guard) frequency, 121.5 MHz. 

PLI is available to aircraft any time pilots are monitoring a common frequency. 
However, the reliability of PLI is not assured. PLI is available only part time since there is 
no certainty that an aircraft which might need the information is tuned to the appropriate 
frequency. Additionally, if the crew is in a high workload situation, they may not have the 
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spare cognitive capacity to monitor PLI communications. Another party line issue is the 
danger of constructing a mental picture of a situation based on incorrect assumptions. 
However, based on cockpit observations, most pilots appear to use PLI only as a 
supplemental information source and rarely make important decisions predicated on PLI 
alone. 


An analysis of ATC communications in the Chicago terminal area (Appendix A), 
indicated that PLI with potentially useful information was present in approximately 90% of 
arrival transmissions. It should be noted, however, that these results are biased towards 
high PLI content since the criteria for potential PLI in the analysis included any 
transmission containing PLI regardless of its importance or context. The PLI content 
relevant to a particular aircraft is normally only a small fraction of the total PLI available on 
the frequency. The fraction of communications that contained potential PLI did not appear 
to vary significantly with different weather conditions or arrival traffic congestion in the 
areas observed. 

2.2. Air/Ground Digital Datalink Development 

2.2.1. VHF Voice Communications 

Currently, air/ground communications between ATC and flight crews are carried 
out almost exclusively by voice radio transmissions that contain all clearance, advisory, and 
warning information. A formalized communication protocol exists between pilots and 
controllers which works reasonably well, however, even strict adherence to these 
procedures does not guarantee successful message comprehension. 

Despite the considerable efforts that have gone into developing current ATC 
communications procedures, significant problems inherent to oral information exchange 
remain. These problems consist of both human and system factors. Human factor 
elements include substandard radio technique and clearance retention problems. For 
example, retention difficulties can occur when controllers attempt to economize “air-time” 
by issuing rapid and complex multi- parameter verbal instructions that can tax pilot’s short 
term memory resulting in erroneous clearance interpretation. System factors include 
frequency congestion problems and simultaneous transmissions which result in the 
frequent blocking of transmissions requiring message repetition, or worse, the 
acknowledgment of a message by the wrong receiver. This creates two problems for the 
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ATC system to resolve. Discrete aircraft addressing using a digital datalink can reduce 
problems of overlapping transmissions on congested channels. The storage of the digital 
information in on-board computers will allow review of complex instructions thereby 
reducing clearance interpretation problems. 

2.2.2. ACARS 

Air carriers have been utilizing air/ground datalink communications for many years 
to efficiently exchange company information such as departure and arrival times. The 
ACARS (ARINC Communications Addressing and Reporting System) system is operated 
by ARINC (Aeronautical Radio Incorporated), a firm set up by several airlines that 
specializes in providing communications between airline operations and aircraft by using a 
network of landlines, phone patch relays, and/or datalink. The ACARS unit is a terminal 
located in the cockpit with which information can be manually downlinked by 
crewmembers. In some cases, specific information elements (such as engine performance 
data) can be automatically downlinked. ACARS currently uses dedicated VHF channels. 
Messages, such as destination weather or arrival gate information, can also be uplinked 
from the ground and routed to an onboard printer. Information exchange can be initiated 
by the flight crew, the airline's operations, or automatically. 

2.2.3. ACARS Pre-Departure Clearance and FMC Programming 

Recent ACARS developments include the ability to receive Pre-departure ATC 
Clearances (PDC) and accomplish on-board Flight Management Computer (FMC) 
programming during preflight planning prior to boarding the aircraft. The PDC program 
began on a trial basis at Chicago O’Hare, San Francisco, and Dallas-Fort Worth, and after 
demonstrating favorable operational effectiveness, is now available at most major U.S. 
airports. To initiate a PDC, the crew requests their ATC clearance via ACARS and 
confirms acceptance, at many airports, by reading back the transponder code (a unique 
code assigned to each flight for positive radar identification) upon initial contact with 
ground control. Clearance confirmation procedures vary among airports. 

The Flight Management Computers (FMC) aboard modem “glass cockpit” aircraft 
require a significant amount of pre-flight programming of the waypoints that define the 
proposed route of flight in addition to critical performance information. For several 
airlines, FMC programming via the ACARS datalink can be initiated by the flight crew by 
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typing a simple code into any company computer terminal. The entire proposed route of 
flight, winds aloft, takeoff speeds, and performance data can be batch transmitted to the 
aircraft’s on board FMC via the ACARS unit, saving as much as 25 to 30 minutes of pre- 
flight programming (in the case of extended transoceanic flights), and reducing pilot input 
errors. 

2.2.4. Datalink and the National Airspace System Plan 

In the proposed ATC datalink system, digital messages will be electronically 
transferred to visual displays or printers located in aircraft cockpits. Other modalities such 
as synthesized voice are also being evaluated [2]. The system may incorporate a broad 
range of VHF channels, including HF, satellite and Mode S. It should be noted that, the 
National Airspace System plan does not call for 100% datalink communications. Voice 
communications will always be available as a backup in the foreseeable future. The full 
extent of what information will be communicated digitally has yet to be determined [3]. 

In the oceanic environment, satellite systems will support digital communications 
replacing the antiquated HF voice reporting system currently in use. In the near future, the 
Oceanic Display and Automation Processing System at the oceanic control centers in 
Oakland and New York will be enhanced by automatic FMC position reports (Automatic 
Dependent Surveillance, ADS) downlinked to FAA facilities via a satellite datalink. The 
expected increases in reliability and accuracy over the current HF system may allow a 
relaxation of the conservative separation standards now in use. This will enable the system 
to accommodate greater numbers of simultaneous transoceanic flights and provide more 
flexibility in flight level and route selection [3]. 

Mode S datalink communications are slated to provide relief to congested terminal 
operations. In addition to ATC surveillance and tracking capabilities of the current Mode C 
system. Mode S acts as a modem for two-way digitally coded data exchange between the 
ground and the aircraft for ATC purposes. The transponder also passes data between 
aircraft for collision avoidance purposes (TCAS, traffic collision avoidance system). The 
Mode S system is expected to serve as the FAA’s primary domestic datalink for the 
delivery of ATC and flight advisory services [3]. 

Potential uses for an ATC datalink system include transmitting alphanumeric route 
amendment messages to aircraft, and the possibility of automatically loading route 
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modifications directly into onboard FMC’s (in a manner similar to ACARS pre-flight 
programming). The flight crew would, of course, retain the authority to accept or reject the 
modified route. Automatic FMC loading would allow the crew to obtain a graphical 
depiction of the proposed amendment and then execute/comply with the clearance within a 
reasonable amount of time. This would help to increase safety margins by reducing the 
amount of “heads down” time crews spend reprogramming during busy phases of flight 
thereby allowing them to spend more time exercising the “traffic watch” required to 
maintain adequate separation from other aircraft. 

2.3. PLI and Datalink Issues and Tradeoffs 

In the datalink environment, the availability of PLI will be reduced due to the 
discrete nature of datalink addressing where an aircraft only receives messages intended 
solely for it Some voice communications will still be present since not all aircraft will be 
equipped with datalink, and datalink-equipped aircraft may from time to time require voice 
transmissions depending on need or established procedures. For the most part, however, 
datalink will provide for a quiet flight deck environment where crewmembers will not need 
to constantly screen the background chatter for useful information. Although pilots learn to 
filter out unrelated information while engaged in other activities, some cognitive resources 
are employed to screen the incoming data and flag the crewmember’s attention when 
needed [4], 

The results of past research indicate that many pilots are concerned with the 
potential reduction of PLI [2, 5, 6, 7,8,9]. Many international pilots draw parallels to the 
loss of situational awareness experienced when operating at foreign destinations where the 
background communications are in an unfamiliar language. The value of PLI must be 
balanced against the detrimental aspects of having to filter large amounts of verbal data in 
order to obtain useful information. In the absence of any form of compensation, it would 
appear that the intrinsic benefits of datalink - accuracy, lack of congestion, automatic FMC 
programming, etc. - are only possible at the expense of a reduction in the available PLI. It 
is therefore necessary to identify important party line elements so protocols and strategies 
for retention and/or compensation can be developed. 

In order to maintain the benefits of PLI but reduce the negative impact on situational 
awareness, care must be taken in designing datalink protocols. Previous attempts at 
compensatory strategies designed to augment situational awareness include CDTI (Cockpit 
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Display of Traffic Information - a predecessor to TCAS [10]) and proximity addressing, 
where messages are datalinked to aircraft within a specific radius of one another. 
Unfortunately both these schemes increased the crew workload and heads down time as 
flight crew members were required to screen data to determine relevancy [9]. 

2.4. Research Focus 

The specific focus of this research has been to attempt to determine the significance 
of PLI to air carrier crews in the current ATC environment in order to provide a baseline 
from which decisions on datalink implementation can be made. Certain elements of party 
line information may contribute significantly to pilot situational awareness and 
consideration should be given to means of preserving the most useful information in the 
datalink environment. Specific issues which were addressed include: 

- Identification of important party line information elements. 

- Determination of the utility of party line information in the current environment. 

An understanding of the amount of utilization of important party line elements 
should provide a basis for preservation strategies to maintain the benefits of PLI in the 
datalink environment Preservation by specified procedures and/or compensation 
technologies require the knowledge of which elements are used and how they are used by 
pilots. In order to investigate these issues, user input was solicited through pilot opinion 
surveys. The survey results were then used to design a full-mission flight simulation 
experiment which studied how important elements were used in cockpit decision making of 
air carrier flight crews. 
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3. Pilot Opinion Survey 


3.1. Objectives 

In order to assess the overall usage of PLI in the current ATC environment, specific 
input from the users was solicited through a pilot opinion survey. The survey was limited 
to active transport category crews in order to focus on the most frequent users of high 
density airspaces that ATC services. The goal of the survey was to obtain subjective user 
data in the following areas: 

- Assessment of the importance of PLI in the current ATC environment 

- Assessment of the accuracy of PLI in the current ATC environment. 

- Assessment of the availability of PLI in the current ATC environment. 

- Subjective opinions concerning datalink: 

• User preference of datalink/PLI environment with and without compensation. 

• User assessment of effectiveness of TCAS as PLI compensation. 

• User opinions on possible PLI compensation strategies. 

Flight crew input on the importance, accuracy, and availability of PLI was solicited 
for information specific to certain phases of flight to determine the variation of PLI 
significance among different flight regimes. Pilot input concerning general information that 
is independent of phase of flight was also solicited. 

3.2. Survey Design 

The survey was organized in three sections. The first section studied the 
importance, availability, and accuracy of specific party line information elements as a 
function of phase of flight. The second section focused on general items and information 
valid across all phases of flight In both of these sections, pilots were presented with 
individual PLI elements to be scored with incremental rating scales in terms of importance, 
availability, and accuracy. In the third section, pilots were asked for their opinions on 
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various datalink implementation issues. This section also contained an area for free 
comments. 

3.2.1. Section / - Phase of Flight Specific PLI 

The first section of the survey solicited pilot input on party line importance, 
availability, and accuracy across six phases of flight from pre-departure to final approach. 
An example of the format of the survey depicting the rating scales for the Departure phase 
is shown in Figure 3.1. 


IMPORTANCE AVAILABILITY ACCURACY 


PHASE OF FLIGHT 


Departure: takeoff to top of climb 
next comm freq 
weather situation 
ride reports/turbulence 
traffic watch 
controller errors 
other 


non- 
trivial critical existent 

1 2 3 4 5 1 2 

1 2 3 4 5 1 2 

1 2 3 4 5 1 2 

1 2 3 4 5 1 2 

1 2 3 4 5 1 2 

1 2 3 4 5 1 2 


common- 




place 

unreliable 

reliable 

3 

4 

5 

1 2 

3 

4 5 

3 

4 

5 

1 2 

3 

4 5 

3 

4 

5 

1 2 

3 

4 5 

3 

4 

5 

l 2 

3 

4 5 

3 

4 

5 

1 2 

3 

4 5 

3 

4 

5 

1 2 

3 

4 5 


Figure 3.1 Survey Sample 


The survey presented potential PLI elements so that each element could be ranked 
for the Importance, Availability, and Accuracy of the information. This format allowed the 
subjects to consider the different aspects of each element simultaneously. The general 
organization of the survey presented these elements according to phase of flight since 
certain elements are found only in specific phases, and significance may vary across 
phases. The subjects were asked to rank each item for importance, availability, and 
accuracy on the 5 point scales defined in Figure 3.1. 


In order to present the pilots with appropriate PLI elements for consideration, a list 
of potentially important party line information elements was constructed. The full list of the 
PLI elements included in the first section of the survey is presented below in Table 3.1. 

The list was developed from a preliminary survey distributed to 40 air carrier pilots. The 
preliminary survey offered various PLI elements which the subjects were asked to rank by 
significance. Those initial elements were selected based on a strawman list of candidate 
PLI elements drawn up by several aviators experienced in air carrier, general aviation, and 
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military operations. In addition to information on PLI significance, data from the 14 
respondents to the initial survey included comments on survey design, question format, 
and additional PLI information elements. These results were used in producing the final 
version of the survey. The final survey is included in Appendix B. A brief explanation of 
the information elements is presented in Table 3.2. 


Ground Operations: dispatch , pre-start , taxi 
Next Comm Freq 
Routing to Runway 
Ground Sequencing 

Taxi “Hold Short” Instructions for Other A/C 
A/C Crossing Active Runway While You Are 
Lined Up for Takeoff 
Controller Errors 


Departure: takeoff to top of climb 
Next Comm Freq 

Weather Situation (including deviations) 
Ride Reports/Turbulence 
Traffic Watch 
Controller Errors 


Cruise: top of climb to top of descent 
Next Comm Freq 

Weather Situation (including deviations) 

Ride ReportsyTurbulence 

Winds Aloft 

Traffic Watch 

Sequencing 

Controller Errors 


Descent: top of descent to 

approach control contact 
Next Comm Freq 

Weather Situation (including deviations) 
Ride Reports/Turbulence 
Traffic Watch 
Sequencing 

Holding Situations/EFC Validity 
Controller Errors 


Terminal Area: approach control contact to 
final approach fix 
Next Comm Freq 

Weather Situation (including deviations) 
Ride reports/turbulence 
Traffic Watch 
Sequencing 

Holding Situations/EFC Validity 
Terminal Routing/Runway Assignments 
Approach Clearance 
Controller Errors 


Final Approach: final approach fix to 
runway threshold 
Next Comm Freq 
Weather Situation (minimums) 
Traffic Watch 
Sequencing 

Missed Approach - weather induced 
Missed Approach - other 
Windshear 

Aircraft on Your Landing Runway 
Braking Action 
Taxiway Turnoff 


Table 3.1 PLI Categories by Phase of Flight 
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Next Comm Freq - Next communication frequency to be used, i.e. next controlling sector. 

Routing to Runway = Sequence of taxiways to follow in order to arrive at runway. 

Ground Sequencing = Sequential order of aircraft on the ground. 

Taxi " Hold Short" Instructions for Other A1C = Instructions to other aircraft from ground control to give 
way to ownship. 

A/C Crossing Active Runway While You Are Lined Up for Takeoff — Other aircraft cleared by tower to 
cross the active runway downfield while ownship is holding in position lined up on the same 
runway awaiting takeoff clearance. 

Controller Errors = Erroneous information and/or instructions issued by air traffic control. 

Weather Situation = Terminal or enroute weather conditions which may result in deviations. 

Ride Reports/Turbulence = Reports by other aircraft of presence of inflight turbulence. 

Traffic Watch = Out-of-the-window vigilance for the purpose of maintaining safe inflight separation from 
other aircraft. 

Winds Aloft = Wind direction and speed at a given altitude. 

Sequencing = Sequential order of inflight aircraft, usually along a common course. 

Holding Situalion/EFC Validity = Determination of number of other holding aircraft and the anticipated 
departure time from the holding pattern. 

Terminal Routing/Runway Assignments = Flight course as aircraft approaches destination airport, including 
active runway for landing. 

Approach Clearance = Clearance from ATC to begin executing published approach procedure currently in 
use for a given runway. 

Missed Approach - Weather Induced = Balked landing because of weather below minimums resulting in the 
inability to visually acquire the runway environment. 

Missed Approach - other = Balked landing for reason other than weather, e.g. traffic on runway, etc. 

Windshear = Sudden change in direction or speed of wind which may result in the deterioration of flight 
conditions. 

Aircraft on Your Landing Runway = Previous arrival unable to clear runway of intended landing resulting in 
the loss of safe separation. 

Braking Action = Aircraft braking ability as effected by contaminated runway surfaces (snow, ice, standing 
water, etc.). 

Taxiway Turnoff = Taxiway to be used to exit runway after landing rollout. 


Table 3,2 Explanation of Party Line Information Elements 
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3.2.2. Section II - General PLI 


The second section of the survey addressed numerous party line items which were 
not related to a specific phase of flight This section also addressed information that is not 
directly related to aircraft operation but may be used by pilots for overall situational 
awareness. The general elements were ranked for importance, availability, and accuracy 
using the same rating scales as Section I. The elements contained in Section II of the 
survey are listed below in Table 3.3. The first four elements are related to prosodic 
information which is indirect information transmitted by voice inflection or phraseology. 

- Controller’ s experience level inferred from tone of voice and speech patterns. 

- Pilot’s (of other aircraft) experience level inferred from tone of voice and speech patterns. 

- Controller’s “level of urgency" inferred from tone of voice and speech patterns. 

- Pilot’s (of other aircraft) “level of urgency” inferred from tone of voice and speech 

patterns 

- Sector congestion (as indicated by frequency congestion). 

- Background ATC transmissions used as reassurance of being “in contact” with the 

controller (“Anybody out there?” ). 

- Call sign confusion (other aircraft accepting your clearance or vice versa). 

- ATC facilities and problems! lost communications. 

- Navaid facilities and problems. 

Table 3.3 Non-Phase Specific Party Line Survey Items 

3.2.3. Section III - Implementation Issues 

In the third section of the survey, pilots were given the opportunity to express their 
views on a potential datalink communications environment and to offer suggestions on the 
effectiveness of possible party line preservation schemes. The first two questions 
contained rating scales where pilots could record their degree of preference for a datalink 
environment, with and without provisions for the compensation of party line information. 
This was done to assess the perceived value of PLI compensation. These were followed 
by a question addressing the use of TCAS as an alternate means for enhancing situational 
awareness. The specific wording of these questions is presented with the results in Section 

3.3. Pilot’s comments concerning compensation or PLI in general were explicitly solicited. 
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A background section was also included to acquire data about the pilot’s flight experience, 
equipment flown, type of training, familiarity with EFTS (Electronic Flight Instrumentation 
System) equipment, and personal computer use. 

3.3. Survey Results 

The survey was distributed to 1500 Chicago O’ Hare-based American Airlines flight 
crewmembers. Authorization was obtained from both American Airlines management and 
the local Allied Pilots Association (APA) representation to place the surveys in the 
individual pilot mailboxes at the company’s operations center. Responses were collected in 
a box located in the same room. All respondents were guaranteed individual anonymity 
and the surveys were kept confidential. A total of 184 surveys were recovered, most of 
which were returned within two months of initial distribution. The response rate of 12% 
was considered normal for a voluntary survey of this type, particularly due its extensive 
nature. However, because the respondents were self-selected, the data may not be fully 
representative of the pilot community at large. 

The survey results are presented below as mean ratings for each PLI element since 
the variance did not change significantly for most cases. However, the complete numerical 
results along with standard deviation data are available in Appendix C. A 95% confidence 
interval analysis on the mean response ratings is shown in Appendix D. 

3.3.1. Section I: Importance Ratings 

The mean value of importance for each element from Section I is depicted in Figure 
3.2. The perceived overall importance of the surveyed PLI elements is indicated in the 
figure by the high mean scores. All but three items scored greater than the midpoint score 
of 3.0. In addition, there is some variation among specific individual elements as a 
function of flight. For example, traffic watch and controller errors scored lower in cruise 
as compared to departure and terminal area. These items, which contain information used 
in aircraft flight path management, tend to be more time critical in dynamic phases of flight 
such as departure or terminal area operations. In contrast, PLI involving strategic 
information such as ride reports, which rated higher in cruise than in climbout or descent, 
appears to be more important in less dynamic flight regimes. The importance for each 
phase of flight as a group is examined in detail in Figure 3.5 later in this section. From an 
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Final Terminal Descent Cruise Departure Ground 

Approach Area Ops 


overall view of the data, it is apparent that those items concerned with the terminal area and 
final approach phases tend to be higher in importance than the other phases. 
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Figure 3.2 Importance of PLI, All Elements 
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As seen in Figure 3.2, the perceived importance of a given party line information 
element varied among different phases of flight To obtain an integrated picture of the 
importance of topical groupings of party line information, the scores of related elements in 
different phases of flight were averaged. The mean importance of the related groups of PLI 
(across all phases of flight) are shown in Figure 3.3. 



trivial 


critical 


Figure 3.3 Importance of Related Groups of PLI Integrated Across Phases of Flight 

From Figure 3.3 it is clear that PLI containing weather information was perceived 
as most important. This category was comprised of both enroute weather and terminal 
windshear. Holding situation and ride report information scored 2nd and 3rd at similar 
importance levels. These top three items contain information used in strategic planning. 
This observation was supported by the following pilot comments: 

“I have personally used such info to avoid potentially hazardous weather and other 
situations including clearances given in error where such info caused me to question 
a clearance relative to what was going on around us." 

“A pilot needs to have a feel’ for how the system is flowing toward his destination. 
Weather, deviations, rides, holding and EFC’s for others are important for a pilot 
so he can ‘look’ ahead." 

Traffic watch and controller errors contain information typically used for tactical 
planning and, although important, scored slightly less than the strategic elements. The next 
communication frequency appeared to be more of a convenience item than an important 
information element as indicated by the neutral importance score of 3.0. 
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In order to focus on the elements perceived as most important, the 10 highest 
scoring PLI elements or related groups are presented in rank order in Figure 3.4. All 
elements or groups in Figure 3.4 scored above 4.0 in importance. 



‘related groups of elements 


Figure 3.4 Importance of Specific PLI Elements 

As can be seen, the top five elements are all “runway” or “near-runway” events. 
Windshear on final approach ranked the highest in perceived importance. This is consistent 
with similar findings in other studies that indicate pilot reports are perceived to be the most 
accurate or reliable source of windshear information [11]. 

The second and third elements in importance were the presence of an aircraft on the 
runway of intended landing and an aircraft crossing the active runway. It should be noted 
that an aircraft on the runway of intended landing was a major factor in an accident which 
occurred at Los Angeles (LAX) on 1 February 1991. Since this occurred during the same 
time frame that the survey was distributed, it may have influenced the perceived importance 
of this element. 

The other runway related elements, braking action and missed approach 
information, ranked 4th and 5th in importance. Windshear, braking action, and missed 
approach information vary quickly in time and are important to flight operations. Pilots 
endeavor to prepare for these conditions by actions such as approach reference speed 
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adjustments, abandoning the approach in the presence of windshear, or selecting a more 
appropriate auto-brake setting during poor braking conditions. 

Terminal routing and approach clearance information ranked 6th and 7th in 
importance, scoring just above the 4.0 level. Both of these items occur in the high 
workload environment near the destination airport. Terminal routing usually culminates in 
the approach clearance from which point pilots fly the published approach procedure to 
landing. Early indications of the expected routing or approach allow pilots to set up the 
applicable navigation systems or program the FMC before entering the busy terminal area 
where these tasks would require undesirable "heads down" time . 

It is important to note that all of the top scoring items are associated either with an 
aircraft’s approach or landing. These phases are typically the most time critical phase of 
flight The time critical nature of arrival activities (including minimizing “heads down” time 
in order to provide for an adequate traffic watch) and the high importance attributed to party 
line information in the terminal area, suggest that, from a PLI standpoint, approach and 
tower control frequencies are less desirable candidates for initial datalink implementation 
than other "enroute" operations. 

In order to investigate the relative importance for different phases of flight, the 
mean score across all items within a given phase was tabulated in Figure 3.5. 



Figure 3.5 


Importance of PLI by Phase of Flight 
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As noted above, there appears to be a weak trend for the perceived importance of 
PLI to increase with proximity to the airport. While this may not be statistically significant, 
this observation was also supported in the general comment section of the survey: 

“The need for PLI varies inversely with distance from a congested terminal area. 
Pilots subconsciously train themselves to filter out unneeded PLI. Data-linked PLI 
will all be presented with the same priority and be more difficult to filter , thereby 
compounding the problem when need is greatest." 

“Many times PLI is misleading - next freq doesn’t apply to us or ride report 
doesn’t apply to our area. ..PLI becomes more critical the closer one gets to the 
field; there almost all info applies to us - holding, next freq, speeds, expected 
runways, go-arounds, runway conditions...” 

In addition, the phases of flight where course and altitude adjustments are frequent, 
such as departure, terminal area, and final approach, generally scored higher than ground 
operations or cruise. 


3.3.2. Section I: Accuracy Ratings 


The results from the accuracy ratings of the survey concerned the perceived 
reliability of party line information to pilots. The mean reliability values for each element 
are depicted in Figure 3.6. 
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Figure 3.6 Accuracy of PLI, All Elements 


reliable 


As can be seen in Figure 3.6, the accuracy means did not change significantly 
between different phases of flight. The overall accuracy mean was 3.66 which indicates 
that the reliability was perceived to be moderately high. An exception was controller errors 
which had noticeably lower reliability scores. Also, the accuracy of the individual elements 


25 


did not change significantly for each phase of flight Controller errors consistently scored 
low in terms of reliability among all phases which indicates pilots do not have confidence 
that controller errors can be reliably detected from party line information. The highest 
scoring PLI elements from Figure 3.6 indicate that the runway incursion items, aircraft 
crossing the active runway and aircraft on runway of intended landing, were considered the 
most reliable, followed by approach and missed approach items. 


The ratings of each similar element in different phases of flight were averaged to 
obtain a picture of the reliability of topical PLI groupings. Category groupings of party line 
accuracy integrated across all phases of flight are depicted below in Figure 3.7 
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Figure 3.7 


reliable 

unreliable 

Accuracy of PLI Categories Integrated Across Phases of Flight 


Among the topical groupings in Figure 3.7, the low score for the reliability of 
detecting controller errors is the most significant. Party line information as a means of 
detecting controller errors is perceived as important (3.9/5.0) by pilots but not very reliable 
(3. 1/5.0). 


Weather situation information scored as most reliable. This is consistent with the 
weather “importance” scores and also supports the finding that weather related reports from 
other pilots (PIREPS) are perceived to be highly reliable. 

3.3.3. Section I: Availability Ratings 


The results of the availability ratings from the survey are shown in Figure 3.8 and 
indicate the perceived availability of PLI. Note that the survey addressed the perceived 
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availability of PLI elements ranging from nonexistent to commonplace. Pilots may 
perceive information to be highly available, even if it is infrequent as long as it is available 
when needed. For example, holding situation information was perceived to be moderately 
available, 3. 8/5.0, even though it is infrequent, because the information is commonly 
available in those situations when holding is occurring. 
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Figure 3.8 Availability of PLI, All Elements 
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The overall availability did not change significantly between different phases of 
flight, although from Figure 3.8, it can be seen that the final approach items tended to rate a 
somewhat higher availability than in cruise. Also, the availability of specific items such as 
traffic watch and sequencing increased during the descent and terminal area phases. This is 
consistent with the tendency for traffic density to increase as approaching aircraft converge 
on a destination. The overall availability mean over all phases was 3.56 indicating most 
items rated as moderately available. The exception was controller errors which had the 
lowest availabilities for each phase of flight. In addition, the low availability of items such 
as winds aloft in cruise suggest that these elements are good candidates for PLI 
compensation. 


Specific types of PLI elements were again grouped into the related categories 
integrated across all phases of flight and ranked by availability. The results are depicted in 
Figure 3.9. 
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Figure 3.9 Availability of General PLI Categories 

The categories of weather situation, ride reports, next communication frequency, 
and hold situation/EFC were considered the most available. In addition to being issued in a 
standardized format, the information contained in all of these items is relatively easy for 
pilots to apply (or extrapolate) to their own flight situations. Weather situation and ride 
reports are commonly requested by crews, therefore much of this information is available 
to all listening on the frequency. Next communication frequencies are issued at high 
frequency in a standard format and are repeated to each aircraft passing out of the sector in 
a specific direction. Holding situation procedures, while less frequent, also require a 
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standard and complete communication protocol including specific entry information and 
expected further clearance times which are easy to interpret. 

Traffic advisories and sequencing vectors, which rated slightly lower than the top 
four, are routinely given to flight crews by ATC, especially during departure and arrival. 
However, in order to interpret the PLI, these elements require an understanding of the 
tactical situation which might not always be available. Controller errors cumulatively 
scored the lowest due in part to the unpredictability of these occurrences and the difficulty 
in identifying errors from PLI alone. 

3.3.4. Section II: General PLI Elements 


The importance ratings from PLI elements in Section II which are not directly 
related to specific phases of flight are shown in rank order in Figure 3.10. 
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Figure 3.10 Importance of Miscellaneous PLI Elements 

Call sign confusion was perceived as the most important general PLI element. 
However, call sign confusion is expected to be alleviated in the datalink environment by the 
inherent nature of discrete datalink addressing. ATC problems/lost communications were 
also perceived as important. It should be noted, however, that the survey responses were 
likely to have been influenced by a major power outage at Chicago O’ Hare (which caused 
widespread arrival problems) during the period the survey was distributed. Controller 
urgency ranked third in importance (first in importance among the prosodic elements), and 
had the highest perceived reliability in the accuracy rating. Controllers typically issue 
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instructions in a calm, professional demeanor and any deviation from this, either by voice 
inflexion or phraseology, is likely to be indicative of a critical situation. Background 
reassurance ranked relatively low but was still considered somewhat important as indicated 
by an above neutral score of 3.31. 


3.3.S. Section III: Compensation Issues 


Flight crew input concerning the PLI/Datalink tradeoff and possible compensation 
strategies was addressed in Section ID. Preferences for a datalink or PLI environment 
were explored by the following questions: 


Considering the advantages of datalink (i.e. quiet frequency, discrete aircraft addressing, 
frequency congestion relief, unambiguous clearances, etc.) and of party line information 
(situational/environmental awareness, traffic! ride information, etc.), would you tend to 
support a datalink environment or the current environment containing party line information 
(PU)? 


Choose one: 

1 2 3 4 5 

DATALINK EQUAL VHF VOICE 

ONLY DISTRIBUTION COMMUNICATION 

VOICE / DATALINK ONLY (PLI) 


If some mechanism could be developed to datalink critical PU to the aircraft (e.g.. a status 
display with current wx, sequencing, and! or holding information), would you tend to 
support a datalink environment or the present PU environment? 


Choose one: 


1 2 3 4 5 

DATALINK EQUAL VHF VOICE 

ONLY DISTRIBUTION COMMUNICATION 

VOICE / DATALINK ONLY (PLI) 


The mean result for the first question was 3.03 with a standard deviation of 0.82, 
indicating a preference for equal distribution of voice and datalink, i.e. the mixed 
environment When the question was reiterated with the qualification that PLI 
compensation would be accomplished by some means of datalinking critical PLI to the 
cockpit, the mean and standard deviation were 2.71 and 0.92 respectively. The average 
“shift” towards datalink (by a single respondent) was 0.50. These results indicate pilots 
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appear to be more receptive to the use of datalink if consideration for the use of 
compensation is given. 


Finally, the TCAS qualified pilots were asked the following question concerning 
the use of TCAS as a compensation system for situational awareness: 

Are you TCAS qualified? YES NO If so, please comment on the 

effectiveness of TCAS as a compensational device for situational awareness, sequencing, 
and traffic watch in the datalink environment. 


The answers to the questions were categorized as follows: 


Favorable: 44 

Neutral: 45 

Unfavorable: 24 


While most of the comments were neutral or somewhat favorable, the large spread of 
opinion can be seen in the following related comments: 


Favorable 

'7 have noticed myself watching the TCAS in the terminal environment to obtain 
sequencing information. The only drawback is that as another aircraft vertically 
separates from you (too high or too low) you lose him, and the information.’’ 


Neutral 

“[TCAS] Should never be used as a substitute for situational awareness. Rather, 
an enhancement to it.” 


Unfavorable 

“No comparison. TCAS has far too many targets when in the most critical phase of 
flight - the terminal area. I personally get a far better situational ‘picture’ by 
listening and being active in ‘the loop’ to ATC. / also find TCAS to be a diversion 
to the highest degree possible. We are too often looking at the TCAS display 
instead of out the window !” 



4. 


Full-Mission Flight Simulation Experimental Study 


4.1. Objectives 

The results of the pilot opinion survey in Chapter 3 indicated that many party line 
elements are perceived as important by active flight crewmembers. In order to study the 
usage of party line information during normal flight operations, a full-mission flight 
simulation experiment was developed. Specific PLI elements with testable responses were 
scripted into the background ATC voice communications. This experiment occurred during 
voice communication flights which were the control phase of a "datalink" experiment 
conducted in the NASA-Ames Man-Vehicle Simulation Research Facility (MVSRF). 
Crewmember responses to each test element were evaluated in terms of level of awareness 
and action taken. The experiment was constructed with the following objectives. 

- Examine the effectiveness of PLI elements perceived as important in the survey responses. 

- Study the assimilation of PLI by examining pilot awareness of scripted PLI events. 

- Study the usage of PLI by examining pilot action responses to scripted PLI events. 


4.1.1. Approach 

The experimental approach was to examine PLI utilization in a high fidelity full- 
mission simulation that presented party line elements in an operational context The 
simulation examined questions involving PLI element utilization including whether or not 
the PLI perceived as important in the survey data could be readily assimilated by the flight 
crew. If assimilated, the study also examined how effectively the information was used. 

Specific PLI elements which rated highly in terms of importance in the survey were 
scripted into a high resolution ATC voice background scenario in a way that would yield 
testable responses. After a training session, qualified air carrier crews were exposed to the 
scripted PLI in a normal operational environment during simulator data runs. The 
responses to the PLI stimulus in each event were observed and analyzed in terms of level of 
awareness and action taken. 
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4.2. Simulation Facility 


The full-mission flight simulation was conducted in the Man-Vehicle System 
Research Facility (MVSRF) at the NASA-Ames Research Center, Moffet Field, CA. A 
major requirement of the simulation study was the ability to reproduce pilot-controller 
dialogue in a high resolution ATC background. The party line elements were scripted into 
the ATC background to study any indications of increased flight crew situational awareness 
and/or responses stimulated by the PLI elements. In order to allow for full voice 
interaction between the subject crews, ATC, and other aircraft, the components of the 
facility consisted of a flight simulator cab and an ATC simulation laboratory. In addition to 
providing standard Air Traffic Control functions, the ATC simulation lab could generate 
realistic communications with other background or “pseudo”-aircraft. The interface 
between the advanced cab, ATC, and the pseudo-pilots is depicted in Figure 4.1. 


SCRRTED ^ DIRECT PH.OT-ATC . _ _ _ , BACKGROUNCVPIJ 

COMMUNICATIONS P TRANSMISSIONS TRANSMISSIONS 



ADVANCED CAB FLIGHT SIMULATOR 
Figure 4.1 Advanced Cab - ATC - Pseudo Pilot Interface 
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4.2.1. Advanced Concepts Flight Simulator 

The Advanced Concepts Flight Simulator (ACFS), or “advanced cab”, is a full- 
motion base, six-axis flight simulator with size and performance characteristics similar to 
the twin engine B757. The cockpit interior layout (Figure 4.3) is similar to modem 2-man 
EFIS/FMC-equipped aircraft. A sidestick at each pilot station is used for flight control 
input instead of the yoke which is used on most aircraft Engine, throttle, wing flaps, and 
landing gear controls are all in typical locations (accessible to both pilots), as are the other 
switches such as the seat belt sign and radio control panels. The Mode Control Panel 
(MCP), through which flight crews control autopilot functions, is identical to the one used 
on B757/B767 aircraft in both operation and cockpit location. The Flight Management 
Computer (FMC), similar to those used in navigation and performance operations in 
current transport aircraft, is accessed through two standard Control Display Unit (CDU) 
interfaces located just above the center console. 

Cockpit instrumentation is presented on ten multi-function displays distributed over 
five vertically oriented CRT’s located in front of the pilot stations. The multi-function 
displays on each CRT are depicted in Figure 4.2. 


CRT #1 

CRT #2 

CRT #3 

CRT #4 

CRT #5 

Flight 

Navigation 


Engine/ 

Systems Status 
Datalink, 
Approach 
Charts 


Advisory, 
Caution, 
and Warning 
Messages 


Electronic 

Checklists/ 

Functional 

Systems 


Flight 

Navigation 


Figure 4.2 ACFS Multi-Function Displays 


The CRT’s accommodate “touch screen” input which is the primary interface mode 
for most systems selection and operation. The electronic Attitude Director Indicator (ADI) 
and Electronic Horizontal Situation Indicator (EHSI) “moving map”, usually presented on 
the outermost CRTs (the CRT displays are interchangeable), are functionally identical to 
most Electronic Flight Instruments (EFIS) currently in use. Similarly, the engine/systems 
status and the advisory, caution, and warning messages are also typical of that available on 
current aircraft Datalink, electronic approach charts, electronic checklists, and the 
operation of aircraft systems by touch screen input, however, do not have widespread use 
in current aircraft operations. The utilization of such equipment necessitated specialized 
training for the subject flight crews. 
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Figure 4.3 Advanced Cab Cockpit Interior 



In general, the advanced cab flight and instrumentation systems were able to 
reproduce most aircraft operations with acceptable realism during the experiment. One 
notable flight instrument limitation was the inability to display simulated weather radar 
returns. Since it was desirable to explore the utilization of weather related party line 
information, a scenario approach that did not rely on weather radar had to be developed. 
Another limitation was imposed by the Flight Management Computer, which was a new 
addition to the advanced cab and had only recently become operational. This resulted in 
some simulator down time to work out implementation problems. An additional FMC 
limitation was the inability to perform automatic holding patterns due to the lack of a 
“holding page” on the Control Display Unit (CDU). Because holding functions are 
commonly available in FMCs, holding scenarios had to be preceded by an FMC failure 
which forced the crew to perform the hold manually. 

The advanced cab is equipped with a full motion base that can simulate varying 
levels of turbulence, including wind gusts. Windshear models of varying intensity can be 
situated at different locations near several airports in the simulation database. For the 
purpose of this experiment, however, the windshear model was limited to a 15 knot 
airspeed loss due to technical difficulties in implementing the motion-base windshear 
dynamics at an airport whose database wind field was not originally programmed to 
accommodate windshear. 

The simulator Phase II visual capability has been described by Chappel and Sexton 
as follows: [12] 

The outside visual scene is a dusk/night, full color, computer generated simulation. 
Visual databases include several veridical cities/airports and generic en route 
databases. Both light points and surfaces are projected; buildings, pavement and 
terrain are modeled. Two other aircraft can be flown within the field of view, a 
Boeing 727 and a light twin. Ground vehicles also may be seen moving about the 
airport. Visibility ranges from 0 to 40 miles. Flight can be conducted below, 
through and above multiple cloud layers of varying density. The visibility and 
cloud conditions can be pre-set in the flight scenario or varied during the flight from 
the experimenter/observer stations. 

4.2.2. ATC Simulation Lab 

The Air Traffic Control (ATC) Simulator subsystem of the MVSRF is a 
hardware/software complex which provides the MVSRF with the capability of simulating 
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the multi-aircraft, multi- ATC position environment that is required to perform high fidelity, 
dynamic, and real-time full mission flight simulations. 

The ATC simulation system is capable of: 

1) Simulating up to four air traffic control positions, with any combination of 
clearance delivery, ground control, tower, approach control, and low or high center 
enroute sectors. In addition, ATIS can be broadcast for the appropriate database 
airports. 

2) Generating a total of 100 aircraft (referred to as “pseudo-aircraft”), in any 
configuration necessary as the experiment requires. These pseudo aircraft are 
“controlled” at four stations directly opposite of each ATC position. 

3) Allowing for the generation of two “visual” or out-of-the-window targets, which 
can be used to create specific traffic conflicts as the experimental situation 
requires. 

4) Communication with an audio distribution subsystem that allows for multi-channel 
voice disguising between the ATC positions and/or the pseudo-pilot positions, and 
the advanced cab. This subsystem allows for discrete frequency assignment for 
each ATC sector, i.e.: if the pilot in the cab is not tuned to the correct assigned 
frequency, no communication between the pilot and the assigned ATC sector will 
exist. 

As depicted in Figure 4.1, a “controller” performed the ATC duties prescribed by a 
script, using the voice disguiser when the aircraft transitioned between sectors. Another 
technician manned the pseudo-pilot station and also utilized the voice disguisers to provide 
the required scripted party line transmissions generated by the other aircraft in the 
simulation. A third technician acted as a liaison between the controller and pseudo-pilot 
stations. This technician also monitored a cockpit-view video monitor and resolved any 
technical (ATC) or logistical problems that were encountered during the simulation. 


37 



4.2.3. Experimenter/Observer Station 


A description of the Experimenter/Observer Station as overviewed by Chappell and 
Sexton is presented below [12]. 

The experimenter controls the simulator through the use of the 
Experimenter/Observer Stations located either in the cockpit or the Experimenter’s 
Laboratory. The flight can be fully automated so that real-time intervention is not 
necessary. This allows each crew to experience the same sequence of events at the 
same time or at the same point in their flight. For example, when the aircraft has 
been flying for 20 minutes or reaches 10,000 feet, turbulence at a pre-defined level 
can be induced. The experimenter may also wish to control the occurrence of 
events at the time of the flight. This can be accomplished by selecting an item from 
an experiment scenario at the desired moment. 

Data collection is also controlled from the Experimenter/Observer Station. The 
specific parameters and the frequency of collection can be tailored for the phase of 
flight. Both discrete events (switch positions, etc.) and continuos variables 
(altitude, airspeed, etc.) are collected as a function of experiment time. In addition, 
all communications within the cockpit and between pilots and controllers can be 
recorded and time-tagged. 


The Experimenter station also acted as the “company” for the purposes of radio 
communications involving dispatch and maintenance information during the flight 
simulation. 


4.2.4. MVSRF Implementation 

Subjects for the experimental study were selected from a volunteer pool of current 
air carrier crews, qualified on autoflight B737-300, B757, and B767 EFIS aircraft. 
Although it was assumed that crews familiar with these aircraft would have the least 
difficulty transitioning to the “advanced cab” procedures, there may have been effects due 
to the differences between the ACFS and the aircraft that the crews were qualified on. 

Minor differences in FMC procedures and unfamiliar equipment such as the sidestick 
controller and automated electronic checklist, may create a level of workload not normally 
present in certain flight regimes. 

The preparation for each flight leg commenced with a briefing during which the 
dispatch paperwork was issued. To maintain a high level of simulation fidelity, the actual 
dispatch paperwork format from the subject's airline was used. The simulation was 
initiated prior to engine start and taxi, and terminated when the aircraft was parked at the 
gate. During the flight, normal ATC procedures and communications were provided by the 
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controllers in the ATC station. Realistic background communications, including the 
required information for each PLI event, was provided by ATC and the pseudo-pilots. 

4.3. Experimental Design 

4.3.1. Method 

The simulation experiment was designed to examine the usage of PLI during typical 
flight operations. To achieve this, candidate PLI elements were selected from those scoring 
highly in importance on the survey discussed in Chapter 3. Only those PLI elements with 
specific testable responses were selected for inclusion into the simulation scenarios. For 
example, weather situation ranked highest in importance among related groups of party line 
elements (across all phases). Therefore a turbulence related PLI weather event was scripted 
into the scenario. This was accomplished by having an aircraft ahead of the subject report 
turbulence. Crew responses were observed to see if any course deviations or other actions 
(such as turning on the seat belt sign, etc.) were made as a result of the PLI stimulus. The 
party line experiment was part of a larger study of datalink implementation. PLI response 
data was primarily taken during control runs where voice communications were used and 
PLI was available. However, some datalink only conditions were also used to compare 
flight path responses with and without PLI available for several test events. 


4.3.2. Scenarios 

Party line events were chosen for inclusion in the experiment based on the survey 
results from Chapter 3. All items that rated above 4.0 in importance were considered, 
however, only those that could reasonably be included in the simulation were selected. For 
example, since weather radar was unavailable in the advanced cab, no precipitation-based 
weather deviation scenarios could be realistically implemented. The nine PLI events that 
were included in the simulation are listed in Table 4.1 along with their corresponding 
importance rank from the survey. 
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Party 

Line 

Event 

Cateeorv 

Importance 

Survey 

Rank 

PLI 

Aircraft holding short at taxiway intersection 

7 

PL2 

Aircraft crossing active runway while subject is lined up 

for takeoff 3 

PL3 

Turbulence and weather deviations 

4 

PL4 

Aircraft on runway of intended landing 

2 

PL5 

Holding EFC validity 

6 

PL6 

Traffic watch while holding 

5 

PL7 

Traffic watch during climb 

5 

PL8 

Aircraft sequencing 

8 

PL9 

Windshear on final approach* 

1 


*this event incorporates two reinforcing party line elements: windshear and missed approach 

Table 4.1 Experimental Party Line Event Descriptions 


Each PLI event from Table 4. 1 was scripted into the simulation. The script was 
developed by a research group consisting of pilots, simulator engineers, and NASA- Ames 
psychologists. It is presented in entirety in Appendix E. The scripted PLI elements are 
briefly summarized below. 


PLI Aircraft holding short at taxiway intersection 

Subject is on a taxiway approaching an intersection with another taxiway. Another 
aircraft visible to the subject crew is approaching same intersection with instructions by 
ATC to "hold short". The intruding aircraft does not acknowledge ATC and continues with 
no apparent intention to stop and give way. 


PL2 Aircraft crossing active runway while subject is lined up for takeoff 

Subject is in position on the active runway waiting for takeoff clearance (in low 
visibility conditions). A crossing aircraft approaching the active runway acknowledges a 
hold short instruction. Subject is then cleared for takeoff. Shortly after the takeoff roll 
begins, the tower clears the other aircraft to cross the active runway downfield of subject. 
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PL3 Turbulence and weather de\’iations 

Subject is nearing top of climb in trail of other aircraft. Preceding aircraft report 
turbulence along subject’s course line at the altitude that subject is climbing to. 


PL4 Aircraft on runway of intended landing 

Subject is "Cleared to land, number 2" behind traffic on short final in low overcast 
weather conditions. Dialogue between the tower and previous landing aircraft concerns 
vacating active runway. The previous aircraft is unfamiliar with airport and unable to clear 
the runway in a timely manner. As subject breaks out of the overcast, he or she encounters 
traffic on the runway requiring a go-around (either self-initiated or as instructed by the 
tower). 


PL5 Holding EFC validity 

Subject is in a holding pattern after executing a missed approach. Transmissions on 
VHF communications channel reveal other aircraft are still executing the missed approach 
and subsequently reporting entering holding patterns above the subject. Other aircraft in 
hold (below) are receiving revised EFC's (Expect Further Clearance times) that, if 
projected to subject, would result in an unacceptable hold time due to a minimum fuel 
situation. If subject does not recognize the problem, they are contacted by dispatch. 


PL6 Traffic watch while holding 

Subject is holding at 9000’ MSL. A "pop-up" VFR light aircraft checks in at 9500’ 
and is informed by ATC that, on its current course, it will "violate holding airspace" which 
would involve the subject. The VFR traffic's radio phraseology reveals "inexperience". 


PL7 Traffic watch during climb 

Subject is climbing out on departure. ATC issues traffic advisory (referring to 
subject) to crossing aircraft above and ahead of subject. The crossing aircraft responds no 
contact" to repeated advisories and the situation deteriorates to a near miss. 


PL8 Aircraft sequencing 

Subject is in the middle of a line of aircraft being vectored around a rectangular 
pattern (downwind, base, final) All aircraft are given near identical speed, heading, and 
altitude instructions (both in front of and behind subject). The controller erroneously 
neglects to turn subject from downwind to base, but does call for the aircraft in sequence 
directly behind subject to turn. If subject does not detect the missed turn, then the 
controller continues to vector the subject to the final approach course. 
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PL9 Wiridshear on final approach 

Subject is on final approach and the previous aircraft on approach reports large 
airspeed deviations due to windshear, with one of them going around. The subject 
encounters an airspeed loss at 900 feet above the ground that may require a go-around. 


The party line events discussed in Table 4.1 were scripted into a 3-leg simulation sequence 
shown in Figure 4.4: 



Flight Lee 

LAX-SFO (diversion to SMF) 


SMF-SFO 

SFO-LAX 


PLI Elements 

PL1 - Aircraft holding short at taxiway intersection 
PL2 - Aircraft crossing active runway 
PL3 - Turbulence and weather deviations 
PL4 - Aircraft on runway of intended landing 

(resulting in a go-around at SFO, holding, and 
diversion to SMF) 

PL5 - Holding EFC validity 
PL6 - Traffic watch while holding 

PL7 - Traffic watch during climbout 

PL8 - Aircraft sequencing 

PL9 - Windshear on final approach 


Note: 

LAX = Los Angeles 
SMF = Sacramento 
SFO = San Francisco 

Figure 4.4 


Party Line Event Distribution 



The flight profiles are presented in Figures 4.5, 4.6, and 4.7. Each scripted party 
line event is labeled (PLI, PL2, etc.) at the appropriate locations on the profiles. 
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altitude * 1 000 




Figure 4.5 PLI Simulation Flight Profile - LEG 1 
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altitude # 1000 



Figure 4.6 PLI Simulation Flight Profile - LEG 2 
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Figure 4.7 PLI Simulation Flight Profile - LEG 3 
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A typical simulation sequence originating from LAX would proceed as follows. 
After being dispatched from LAX to SFO, the crew performs a normal preflight and 
startup. While taxiing to the active runway, the subjects encounter the ground taxiway 
intersection traffic described in PL1. Once reaching the active runway, the crew is 
instructed to taxi into position and hold, after which the traffic conflict on the active runway 
from PL2 is invoked during the subject's takeoff roll. During the climb to cruise, the 
turbulence report described in PL3 is transmitted. The flight proceeds normally until 
reaching the SFO terminal area. On short final during the approach to landing, the crew 
encounters the disabled aircraft blocking the runway (PL4) and is forced to go-around and 
enter a holding pattern. An FMC failure forces the crew to execute the hold manually 
(since the holding page on the advanced cab CDU was unavailable). The holding EFC 
validity (PL5) and the traffic watch while holding (PL6) events occur during the hold and a 
diversion to SMF is made after the determination that the SFO airport will be closed 
indefinitely. The short flight to SMF occurs without incident and the crew makes a normal 
arrival and landing. 

The crew is then dispatched from SMF to SFO and proceeds with a normal start- 
up, taxi out, and takeoff. During climbout, they encounter the crossing traffic described in 
PL7. The remainder of the flight to SFO occurs without incident and terminates in a 
normal arrival. 

On the last leg, the flight is dispatched from SFO to LAX. A normal start-up, taxi, 
and takeoff occur, however, during climbout the background ATC contains multiple 
information elements indicating that at least two other aircraft (one ahead, one behind) are 
on the same route as the subject. This situation continues throughout the flight during all 
ATC route and altitude amendments, as well as frequency change handoffs. Shortly after 
arrival into the LAX terminal area, PL8 occurs when the controller neglects to issue an 
expected vector to the subject. After being re-vectored, cleared for the approach, and 
switching to the tower frequency, windshear related airspeed loss advisories from previous 
arrivals are transmitted, resulting in PL9. The crew either elects to make a normal landing 
despite the windshear conditions, or executes a missed approach and is subsequently 
vectored to a normal landing. 
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To counterbalance learning effects, each crew flew one of the following flight leg 
rotations initiating in either LAX or SMF. 

LAX-SFO(divert to)-SMF SMF-SFO 

SMF-SFO OR SFO-LAX 

SFO-LAX LAX-SFO(divert to)-SMF 


4.3.3. Measurements 

The experimental objectives were investigated using observational, flight 
performance, and subjective measures. Audio and video recordings were made to monitor 
intra-cockpit conversation and note any explicit responses to the PLI provided, including 
decision making and/or increased levels of preparedness. Flight path and control input data 
were recorded to investigate any difference between cases with and without PLI. 
Observations were recorded by experimenters located in a separate room containing a video 
monitor. Information on the monitor included three different camera angles of the 
simulator interior: captain’s flight displays, first officer’s flight displays, and a wide angle 
view. Resolution was clear enough to observe any switch movements, but not clear 
enough to read the onboard status displays. Other information displayed on the video 
monitor included: airspeed, altitude, heading, vertical speed, current checklist, flap 
position, and a running chronometer indexed to the beginning of the flight leg. Audio 
information contained both intra-cockpit and ATC communications. Subjective opinion 
measures were obtained during an out-interview where subjects were asked to rank and 
comment on the importance of PLI in their performance and responses to the various 
scenarios. 

For the data collection runs, the crews were alone in the simulator. However, if 
any technical or "maintenance" question arose, including simulator problems, they were 
provided a "company" frequency with which they could obtain information. In the 
simulator control room, the simulator engineers monitored and recorded all video, audio, 
flight and status displays, switch positions, and flight path data* from before the first 
checklist was read to after the final checklist was complete. 


‘Switch position and flight path data included: altitude, airspeed, groundspeed, geographical position 
(latitude and longitude), flight control input, engine power settings, flaps, landing gear, radios, and all other 
switch positions. 
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4.4. Experimental Procedure 


Each experiment lasted 1 and 1/2 days. The experimental protocol called for 1/2 
day of training (4 hours) in procedures of the advanced cab in the MVSRF, a full day of 
data runs (8 hours), and out-interviews (1-2 hours). Crews were subject to one of two 
different patterns based on when they were scheduled to arrive at the simulation facility. 

Morning arrival 

day 1 - 1/2 day training and 1/2 day data runs 
day 2 - 1/2 day data runs 

Afternoon arrival 

day 1 - 1/2 day training 
day 2 - full day of data runs 

The two counterbalanced rotations were distributed as evenly as possible among the 
crews starting in the morning and those starting in the afternoon. Also, a short break was 
taken between each flight leg during the data runs. 

4.4.1. Training 

Upon arrival, the subjects were given 1/2 day of training in the advanced cockpit. 
This consisted of difference training on the advanced cab systems, basic flight maneuvers, 
visual and instrument approaches, and navigation to SFO from SMF. The subjects were 
also given a basic briefing on advanced cab systems, electronic checklist, and datalink 
communications procedures. During the training sessions, an instructor pilot was present 
in the simulator with the crews. The flight maneuvers allowed the crews to become 
familiar with the sidestick control and to get a "feel" for the flight dynamics. The visual 
and instrument approaches provided an opportunity for the crews to become familiar with 
the simulator visual display (view outside) and enabled them to practice approaches and 
landings, and tracking the ILS. The navigation exercises allowed the crew to familiarize 
themselves with the advanced cab FMC system and CDU programming. 

4.4.2. Experimental Run 

The subjects participated in the three leg simulation while crew response was 
monitored through audio, visual, and flight path data and observation notes. The 
responses to each party line element for each crew were analyzed according to level of 
awareness and the results of all crews were grouped and ranked. 
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4.4.3. Post Experiment Out-Interviews 


Post experiment out-interviews were conducted to obtain subjective responses to 
the scenarios and the advanced cab, and to examine differences between perceived and 
actual reactions. These interviews also provide insight into reasoning behind responses 
and subjective preferences. 

4.5. Results 

4.5.1. Subjects 

Seven air carrier crews participated in the experiment One "leg" of data was 
missed from one crew due to technical difficulties (and resulting scheduling problems) with 
the MVSRF. Also, a PL2 event from one crew was discarded because of a controller- 
script error. The subjects consisted of 8 captains and 6 first officers with a mean flight 
experience of 9086 hours. The subjects were employed by the same air carrier since flight 
crews from the same airline tend to use standardized procedures that are independent of 
aircraft type. Those chosen to participate in the experiment were guaranteed individual 
anonymity. 

There were two mixed parings within crews, (e.g. a 737-300-qualified captain flew 
with a 757 -qualified first officer, etc.). Mixed crews often resulted in one crewmember 
being more familiar with the advanced cab systems because the procedures of different 
aircraft vary. Consequently, the resulting proficiency imbalance may have subjected the 
mixed crews to a difference in overall workload level which may have influenced the 
results. 


4.5.2. Analysis of Events and Scoring 

Each PLI event was analyzed to determine if the scripted party line information 
resulted in any change in crew level of awareness and if the information elicited any action. 
Consideration was given to intra-crew and ATC-crew discussion, a change in aircraft 
system status, increased preparedness for an anticipated situation, or aircraft flight path 
adjustments made in response to other aircraft/controller voice transmissions. Subject 
awareness and reactions for each PLI event were determined from the following sources: 
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1) Observation notes 

As many as three NASA-Ames researchers recorded observation notes of 
displayed video and audio information while the experiment was in progress. 

2) Intra-cockpit and ATC audio track analysis 

Audio track content of conversation among the subjects as well as communications 
between the subjects and ATC was recorded. 

3) Video Tape Analysis 

Video Tape content of crew actions such as increased out-of- the- window vigilance 
and switch and control actuation. 

4) Discrete switch and control positions 

All switch and control positions were recorded as well as flight control stick force 
and direction input. 

5) Flight path data 

Aircraft positional, altitude, velocity, and acceleration data were recorded. 


The responses to each party line event stimulus were scored based on observations 
using the criteria shown in Figure 4.8, which spanned the range from (1) no response to 
(5) positive action taken. The incremental values (2,3, and 4) corresponded to increasing 
levels of awareness of the event within non-action responses. 


SCORE 


NOT AWARE No indication of response to transmissions 1 

AWARE Passive awareness, no indication of detailed 2 

understanding of event. 

General awareness, indication of detailed understanding 3 

but no action taken. 

Passive action, crew discussion of alternate course of action 4 

and/or increased preparedness. 

ACTION TAKEN Change of system status, flight path adjustment or query 5 

to ATC in response to Party Line stimulus. 

Figure 4.8 Crew Response Scoring Criteria 


The primary analysis was completed by an experienced B767 first officer and spot 
checked by a second experienced pilot. The analysis was also reviewed by several NASA- 
Ames psychologists. The scoring methodology and complete analysis of all events are 
contained in Appendix F. 
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4.5.3. Results of Party Line Event Analysis 


Examination of the video, audio and observation data revealed certain responses 
that were common for each party line event. 

PL1 Aircraft holding short at taxiway intersection 

For the aircraft holding short at taxiway intersection event, most crews increased 
their out-of-the-window vigilance, while two crews went so far as to stop the aircraft until 
the matter was clarified. 


PL2 Aircraft crossing active runway while subject is lined up for takeoff 

No crews detected the aircraft crossing the active runway. These observations were 
supported in the out- interview briefings. 


PL3 Turbulence and weather deviations 

Turbulence and weather deviations typically resulted in the crews turning on the 
fasten seat belt sign, with some crews querying ATC: “where’s that aircraft?” in response 
to a turbulence report by traffic ahead. 


PL4 Aircraft on runway of intended landing 

All of the crews engaged in discussion concerning the aircraft on runway of 
intended landing and increased their preparedness for a possible go-around. However, all 
crews executed go-arounds only after being instructed to do so by the tower, i.e. no go- 
arounds were self-initiated. 


PL5 Holding EFC validity 

In the holding EFC validation event, most crews made the decision to divert 
immediately after the hour and a half holding delays were issued by ATC to other aircraft 
holding below. 


PL6 Traffic watch while holding 

In the holding pattern, crews were occupied with the task of considering diversion 
options and fuel quantity status and consequently most were unaware of the traffic 
approaching their position. Those that were aware, however, appeared to cue on the 
phrase “violating holding airspace” issued to the encroaching traffic by ATC. 
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PL7 Traffic watch during climb 

In the traffic watch during climb event, most crews increased their outside vigilance 
as they attempted to make visual contact with the approaching traffic based on the ATC 
transmissions to the other aircraft. 


PL8 Aircraft sequencing 

In the aircraft sequencing event, many of the crews were aware that they were in a 
sequence of arrivals, but none detected the missed turn or questioned ATC. 


PL9 Windshear on final approach 

All of the crews responded to the reports of windshear on final approach issued by 
previous arriving aircraft In addition to discussing a possible go-around, they made 
approach reference speed adjustments and in many cases reviewed the windshear escape 
maneuver. The responses occurred after switching to the tower frequency where the 
windshear PLI was available. No action was taken based on the airport ATIS information 
which was obtained earlier in the flight while the crew was still monitoring approach 
control. The ATIS information contained no windshear data. 


After rating each party line element as described in Section 4.5.2, the responses of 
all crews are summarized in Table 4.2 in the three broad categories of not aware, aware, 
and action taken. 


PLI 

F.VENT # 

1 Aircraft hold short at taxiway intersection 

2 Aircraft crossing active runway 

3 Turbulence and weather deviations 

4 Aircraft on runway of intended landing 

5 Holding EFC validity 

6 Traffic watch while holding 

7 Traffic watch during climb 

8 Aircraft Sequencing 

9 Windshear on final approach 


NOT 

AWARE 

AWARE 

ACTION 

TAKEN 

1 

4 

2 

6 

0 

0 

1 

1 

5 

0 

7 

0 

0 

1 

6 

5 

2 

0 

1 

6 

0 

1 

5 

0 

0 

0 

6 


Table 4.2 Party Line Event Results Summary 
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4.5.4. Analysis of Relative Party Line Event Responses 


In order to assess which elements were most effective in inducing action responses, 
the scores from Table 4.2 were ranked by action taken in Table 4.3. 


PLI 

EVENT 

NOT 

AWARE 

AWARE 

ACTION 

TAKEN 

Windshear on final approach 

0 

0 

6 

Holding EFC validity 

0 

1 

6 

Turbulence and weather deviations 

1 

1 

5 

♦Aircraft hold short at taxiway intersection 

1 

4 

2 

♦Aircraft on runway of intended landing 

0 

7 

0 

♦Traffic watch during climb 

1 

6 

0 

♦Aircraft Sequencing 

1 

5 

0 

♦Traffic watch while holding 

5 

2 

0 

♦Aircraft crossing active runway 

6 

0 

0 

* = Traffic related 

Table 4.3 Ranked PLI Event 

Results 



Winds hear, holding EFC validity, and turbulence and weather deviations rated the 
highest and all resulted in action responses from the subjects. This finding is in agreement 
with the results from the “importance” section of the survey where the same items rated 
among the most important. The high crewmember response to windshear reports, 
especially if they include previous go-arounds, indicates possible areas for procedural 
changes. For instance, since many crews appeared to cue on the reports of the go-around 
by the previous arrival, it may be advantageous for ATC to simply report previous go- 
arounds as a standard procedure without the need to speculate on cause or wait for a formal 
pilot report (PIREP). 

The top three elements are all strategic events where pilots often have time available 
to make decisions. In the case of windshear, strategic planning includes exercising the 
necessary precautions to completely avoid the windshear or adjust flight parameters to 
minimize any adverse windshear effects. 
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The lowest awareness occurred for the bottom two events, PL2 - aircraft crossing 
the active runway and PL6 - traffic watch while holding. For these two events the amount 
of PLI assimilated was minimal. Both occurred in high workload situations where the 
crew's attention and cognitive capacity was absorbed by the tasks at hand, specifically, 
setting the takeoff power and considering diversion options while holding. The ability for 
flight crewmembers to assimilate PLI appears to vary with workload, time available, and 
the strategic/tactical situation. There appears to be greater PLI utilization for strategical 
decision making in low workload conditions, and low PLI assimilation in short term high 
workload tactical situations like takeoff roll (J PL2 ). In this case, the party line information 
was transmitted when the crew's attention was focused on other duties. 

The items indicated by an asterisk (*) in Table 4.3 are all tactical traffic related 
events. They all elicited very low action responses by the subjects. These items are 
normally the responsibility of the air traffic controller and pilots are only required to "take 
action" in unusual circumstances. Because of the dynamics of the professional relationship 
between flight crewmembers and air traffic controllers, pilots are often reticent to insinuate 
that a controller error has taken place. The low number of action responses may also be 
due to the low level of confidence pilots have in using PLI for detecting controller errors as 
indicated in the survey. As a consequence, the crews were reticent to take action but often 
were aware and did indicate increased vigilance. An additional factor may be that action 
based on an incorrect interpretation of PLI in some traffic related cases had a higher penalty 
than the non-traffic relevant events. For example, in the case of weather deviations (a 
“non-traffic” event), the crew was required to get approval from the controller. Therefore, 
even if the deviation was unnecessary, no serious penalty was incurred since permission to 
execute the deviation was obtained from ATC. 

For traffic events such as aircraft sequencing, crews often receive spacing vectors 
that might seem inappropriate from their vantage point, but rarely question the ATC 
instructions under the assumption that the controller has the "big picture" and any 
disruption might impede the smooth flow of traffic or cause an aircraft separation problem. 
This may explain the experimental observation that PLI utilization in cases involving 
scripted controller oversights (such as PL8) rarely resulted in any positive action taken by 
flight crews. 

The high action responses of the terminal area or “near airport” party line events 
from the simulation study ( windshear , holding, and airport ground events) are consistent 
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with the high importance and accuracy ratings these items had in the survey results. These 
results reinforce the need to proceed cautiously when considering datalink implementation 
in the terminal area and final controller segments. 


4.5.5. Flight Path Analysis 

Flight path data was compared between PLI (voice) and non-PLI (datalink) cases to 
examine any significant differences. The Party Line Event PL4 traffic on runway of 
intended landing was examined where the flight path performance during the controller 
initiated go-around was analyzed. 

In both voice and datalink cases the subjects were vectored for the approach in an 
identical manner. In the voice case, a conversation between a previous arrival and the 
tower concerning clearing the active runway was transmitted over the tower frequency 
during the subject’s final approach. In both cases, as the approach continued, the subject 
broke out of the overcast and was presented with simulated visual traffic on the runway. 
Since no crews self-initiated a go-around, the tower instructed them to do so at the same 
point in the approach. In each case, the time and altitude when the go-around instructions 
were issued were used as a common datum for analysis of the subsequent flight paths. 

The data collected included the time and altitude (tj, hi) that the go-around 
instructions were issued by ATC, and the time and altitude (t2, h2) of the point of minimum 
height above the ground (the distance between the bottom of the flight profile arc and the 
ground). For the datalink case, the issuance of the go-around instructions by ATC was 
defined as the time of the datalink chime (used to alert the crew of the arrival of a datalinked 
message) for the “go-around” datalink message. For the voice case, the time of issuance 
was defined as the time when the controller began transmitting the go-around instructions 
to the flight crew. The results are presented in Table 4.4 along with the differences in time 
and altitude (td, hd) between the “go-around” instruction point and the minimum altitude 
point. These correspond to the crew response time (td) and altitude lost during the go- 
around maneuver (hd). The time units are seconds indexed to the beginning of the 
simulation and the altitude units are feet above the ground. 
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CREW 

tj (sec) 

t 2 (sec) 

td(sec) 

h i (ft) 

h 2 (ft) 

h^ (ft) 

Voice 1 

6285 

6289 

4 

285 

261 

24 

Voice 2 

6758 

6764 

6 

295 

249 

46 

Voice 3 

6607 

6612 

5 

312 

281 

31 

Voice 4 

5991 

6000 

9 

355 

275 

80 

Voice 5 

6740 

6748 

8 

319 

250 

69 

Voice 6 

6029 

6033 

4 

191 

153 

38 

Datalink 1 

5747 

5758 

11 

475 

372 

103 

Datalink 2 

21460 

21468 

8 

286 

208 

78 

Datalink 3 

5923 

5933 

10 

356 

277 

79 

Datalink 4 

6149 

6158 

9 

299 

217 

82 

Datalink 5 

6755 

6762 

7 

282 

246 

36 

Datalink 6 

5929 

5934 

5 

297 

262 

35 


Table 4.4 Voice/Datalink Flight Path Data 


The means and standard deviations from Table 4.4 are as follows. 



Mean 

Standard Deviation 

Voice Time to Go-around, tj 

6.0s 

2.1 

Voice Altitude Lost in Go-around, h^ 

48.0s 

22.1 

Datalink Time to Go-around, tj 

8.3s 

2.2 

Datalink Altitude Lost in Go-around, h,j 

68.8s 

27.4 


As can be seen from the results, there exists an average 2.3 second difference in 
response times between the voice and datalink cases. The slower response in the datalink 
cases is thought to be due to three possible causes: (1) the time required for the crews to 
read the datalinked go-around message, (2) the influence of prosodic voice inflection in the 
verbal go-around instructions, and (3) the enhanced preparedness resulting from the 
awareness provided by PLI (note: all voice crews indicated awareness of the aircraft from 
PLI data). 
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The relative importance of the datalink reading delay, and the influence of PLI and 
prosodic effects cannot be resolved from this preliminary study. However, the presence of 
the 2.3 second lag in the datalink case indicates that for some time critical ATC instructions, 
voice transmissions offer potential time advantages over datalmked transmissions. This 
observation suggests consideration should be given for the retention of voice in the datalink 
environment for time critical or emergency communications. 



5. Summary of Conclusions 


A pilot opinion survey was distributed to study the perceived importance of party 
line information and a flight simulation experiment was conducted to investigate PLI 
assimilation and usage in pilot situational awareness. The major conclusions of this 
research are summarized below. 

Conclusions from the Pilot Opinion Survey: 

1 . PLI is perceived as important by flight crewmembers as indicated by the high mean 
importance scores in the survey. The perceived importance of a given party line 
information element varied among different phases of flight. Those items 
concerned with the Terminal Area and Final Approach phases tended to be higher in 
importance than the other phases. 

2. PLI containing weather situation information was perceived as most important, 
followed by holding information. Communication related general PLI elements 
such as call sign confusion and controller urgency were also perceived as very 
important. 

3. The top scoring importance items from the survey are associated with an aircraft’s 
arrival, which is often the most time critical phase of flight. The time critical nature 
of arrival activities (including minimizing “heads down” time in order to provide for 
an adequate traffic watch) and the high importance attributed to party line 
information in the terminal area, suggest that the final tower controller frequency is 
a less desirable candidate for initial datalink implementation than other "enroute" 
operations. 

4. The elements involving strategic planning: weather situations, holding situations, 
and ride reports scored higher in importance. Elements which contain information 
typically used for tactical planning such as traffic watch and controller errors, 
were also rated as important, but scored somewhat lower. 
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5, Party line information which allows detection of controller errors is perceived as 
important by pilots but not very reliable. Controller errors consistently scored 
lowest in terms of perceived reliability among all phases in the survey which 
indicates pilots do not have confidence that controller errors can be reliably detected 
from party line information. 

6 . Pilots appear to be more receptive to the implementation of datalink if consideration 
for the use of compensation is given. 


i Itj 


Conclusions from the Full Mission Flight Simulation: 

1 . The three PLI elements which scored highest in the survey ( windshear , holding 
EFC validity, and deviations for turbulence or weather) all resulted in action 
responses from the subjects during the simulation study. These are all situations 
where pilots had time available to assimilate information and make decisions 
(action in the windshear event consisted mostly of increased preparedness and 
planned approach speed adjustments made during the pre-approach phase when 
time was available). 

2. The lowest assimilation of the party line information occurred for the aircraft 
crossing the active runway and the traffic watch while holding events. Both events 
occurred in high workload situations where the crew's attention and cognitive 
capacity was absorbed by the tasks at hand, specifically, setting the takeoff power 
and considering diversion options while holding. 

3 . PLI appeared to have greater utilization for strategical decision making in low 
workload conditions, and not in short term high workload tactical situations like the 
takeoff roll. 

4. In the flight simulation study, PLI was utilized to a lesser extent in those decisions 
involving limited options or where the penalty for an incorrect interpretation was 
greatest. 

5 . PLI utilization in cases involving potential controller oversights rarely resulted in 
any positive action taken by flight crews during the simulation. This is thought to 
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be due to the dynamics of the ATC-flight crew professional relationship since pilots 
were often reticent to insinuate that a controller error had taken place. 

6. Datal ink-only crews responded an average of 2.3 seconds slower than the voice 
only crews to a “go-around” instruction issued by ATC in response to an aircraft on 
the runway. All of the voice-only crews indicated awareness of the situation from 
PLI, however other factors such as the time to read the datalink message and 
prosodic effects may also have influenced the results. Still, the presence of a 
notable lag in the datalink case indicates that for some ATC instructions, voice 
transmissions offer certain time advantages over datalinked transmissions. This 
observation suggests consideration should be given for the retention of voice in the 
datalink environment for time critical or emergency communications. 

7 . The high utilization of PLI for the terminal area or “near airport” party line events 
in the simulation study ( windshear , holding, and airport ground events) was 
consistent with the high importance of these items in the survey. The importance 
and utility of PLI near the airport indicates the need to proceed cautiously when 
considering datalink implementation in the terminal area and final controller 
segments. 


60 



REFERENCES 


1. Lee, A.T. and Lozito, S., Air-Ground Information Transfer in the National Airspace 

System, Proceedings of the Fifth Symposium on Aviation Psychology, April 17- 
20, 1989. 

2. Corwin,W.H. and Miles, W.L., Human Factors Considerations of Data Link, 

McDonnell Douglas, 1990. 

3. FAA AC-20-XX, Airworthiness Approval of Airborne Data Link Systems, 1990. 

4. Kems, K., Data Link Communication Between Controllers and Pilots: A Review and 

Synthesis of the Simulation Literature, MITRE, 1990. 

5. Boucek, G.P., Human Engineering Issues for Datalink Systems, SAE G-10 Flight Deck 

Information Management Subcommittee (Seq No. BA010), February 1990. 

6. Chandra, D., An Evaluation of Automation for Flight Path Management in 

Transport Category Aircraft ”, MS Thesis, Department of Aeronautics and 
Astronautics, Massachusetts Institute of Technology, Cambridge, MA, 1989. 

7. Knox, C.E. and Scanlon, C.H., Flight Tests Using Data Link For Air Traffic Control 

and Weather Information Exchange, 1990 SAE Aerospace Technology Conference 
and Exhibition, October 1990. 

8. Lee, A.T., Display-Based Communications for Advanced Transport Aircraft, NASA- 

Ames, 1989. 

9. Waller, M.C.and Lohr, G.W., A Piloted Simulation Study of Data Link ATC 

Message Exchange, NASA Technical Paper 2859, 1989. 

10. Howell, J.D., Simulator Evaluation of Pilot Assurance Derived from an Airborne 

Traffic Situation Display, Report No. FAA-EM-72-3, February 1972. 

11. Wanke, C.R., and Hansman, R.J., Operational Cockpit Display of Ground-Measured 

Hazardous Windshear Information, MIT Aeronautical Systems Laboratory Report 
ASL-90-4, May 1990. 

12. Chappell, S.L. and Sexton, G.A., Advanced Concepts Flight Simulation Facility, 

Applied Ergonomics, December 1986. 

13. Devore, J.L., Probability and Statistics for the Engineering and the Sciences, Cole 

Publishing Co., Monterey, CA, 1982. 


61 



Appendix A: 

ATC Communication Analysis 



A. 3. Results 


Recordings were made between September, 1990 and March, 1991, during peak 
and off-peak traffic conditions and in various weather conditions . Each tracked flight 
resulted in approximately 20 minutes of recorded air/ground communications and included 
the three sequential sectors: enroute center control frequency, approach control frequency, 
and the control tower frequency. 

Analysis of Recordings 

The recordings of the VHF air ground communications were analyzed for type and 
quantity of PLI present. The tapes were reviewed by an experienced pilot and checked by a 
second pilot. While the tapes were being reviewed, the arrival sequence of every aircraft 
on the frequency was noted and each two-way communication interaction or “transaction” 
between ATC and any flight on the frequency was categorized by type according to Table 
A.l. The “General” list was applied for all communications while the ’Tower/ 

Departures” list only involved tower frequency communications which included both 
arriving and departing aircraft Note, with the exception of the last item, all items of both 
lists are considered potential PLI. All of the communications were categorized using the 
groups in Table A.l, however, any communications that did not have the potential to be 
used as PLI by the tracked flight was so identified. 

General 

Next comm fireq 

Sequencing (routing, altitude, airspeed) 

Informational transmissions 
Weather situations 
Approach clearance 
Traffic watch 
Requests 
Ride reports 
Holding situations 
Sector check in 

Miscellaneous non-PLI transmissions 
Table A.l Pilot/ATC Transaction Categories 


Tower/Departures 
Landing clearance 
Landing wind check 
Hold short instructions 
Takeoff clearance 
Position and hold instructions 
Runway exit/ground control contact 
Departure control handoff 
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The communication transactions were then tallied and ratios of potential PLI 
transmissions to total transmissions were determined. There is some ambiguity to defining 
what is pertinent PLI for a given flight. For example, it is not clear whether sector check in 
or ride report PLI from an aircraft sequentially behind the tracked flight provide any useful 
information. For this analysis, potential PLI was defined as any information which could 
be useful as PLI. This then defines an upper limit of available PLI 


The results of all the recorded flights, including the weather and total transaction 
count, are presented in Table A. 1 . For three of the flights the center sector communications 
were unavailable due to reception problems. The ratios for each sector contain all potential 
PLI transmitted, including those involving aircraft sequentially behind the tracked flight. 
Therefore, the ratios are biased towards an upper limit of potential PLI. 


Flight # 

Weather 

Total TX's 

Center 

Approach 

Tower 

All Sectors 

UA69 

38BKN 

1 1 5 

0.67 

1.00 

0.79 

0.91 

AA391 

38BKN 

1 58 

0.89 

1.00 

0.77 

0.95 

AA277 

VFR 

1 1 8 

0.95 

1 .00 

0.86 

0.96 

UA77 

CAVU 

38 

0.85 

1 .00 

0.75 

0.89 

AA783 

CAVU 

1 24 

0.71 

1.00 

0.79 

0.80 

AA41 

CAVU 

43 

0.90 

1.00 

0.71 

0.86 

UA955 

70VC 

82 

1 .00 

1.00 

0.80 

0.98 

NW173 

70VC 

1 1 9 

1 .00 

1.00 

0.83 

0.98 

UA883 

110VC 

63 

1 .00 

1.00 

0.80 

0.98 

SI156 

RVR20 

81 


1.00 

0.75 

0.94 

AA303 

RVR20 

45 


1 .00 

0.86 

0.98 

AA175 

W5S 

84 

0.78 

1.00 

0.52 

0.76 

UA765 

W5S 

128 

0.71 

1 .00 

0.65 

0.80 

AA341 

XM120VC 

90 

0.47 

1.00 

0.69 

0.73 

UA63 

BS-MVFR 

37 


1.00 

0.80 

0.97 

SAB563 

M19BKN 

76 

0.72 

1.00 

1.00 

0.91 

AA581 

VFR 

90 

0.80 

1.00 

0.75 

0.88 

UA81 

VFR 

86 

0.95 

0.94 

0.73 

0.92 

AA53 

VFR 

65 

0.60 

1.00 

1.00 

0.91 

AA783 

VFR 

88 

0.88 

1.00 

0.87 

0.92 

AVG 


86.5 

0.81 

0.99 

0.79 

0.90 

SDEV 


33.03 

0.15 

0.01 

0.1 1 

0.08 


Table A.2 PLI Communications Ratios 
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As can be seen, the overall percentage of potential PLI transmissions was 
approximately 90%. The ratio of potential PLI to total transmissions for the center sector is 
somewhat lower due to the larger number transient (non-arriving) aircraft. In the approach 
control sector almost all of the communications contained potential PLI. Most of the non- 
PLI transmissions from the tower sector included frequency change instructions given by 
the tower to departing aircraft. If these were to be included in the tower ratios, then almost 
all of the communications on this sector could be classified as potential PLI also. 

Peak and off-peak traffic periods are indicated by the number of total transmissions 
for a given tracked flight (since the recording time is approximately the same for each 
arrival). There does not appear to be any significant trends in the fraction of transmissions 
containing potential PLI with respect to weather conditions or peak/off-peak traffic 
conditions. However, if the amount of useful or critical PLI transmissions in a sector 
increased, the effect of weather and traffic conditions may be more pronounced. 
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Appendix B: 

Survey of Datalink ATC Message Exchange 
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DEPARTMENT OF 

AERONAUTICS AND ASTRONAUTICS 


CAMBRIDGE, MASSACHUSETTS 02139 
ROOM 37438 (617) 253-0993 

TELEX 92-1473 FAX (617) 258-7566 


Fellow ORD Crewmember: 

I am currently an ORD based 7671 F/O (Boston commuter). While I am not flying 
(which I usually try to do on weekends), I attend graduate school full time in the 
Department of Aeronautics and Astronautics at MTT where I am pursuing a Master’s degree 
in Aeronautics. In addition to my classes, I work in the Aeronautical Systems Laboratory 
as a research assistant where I am carrying out the research for my Master’s thesis . The 
research involves human factors issues in the implementation of ATC datalink message 
exchange. As you may know, the FAA’s National Airspace Plan for the 1990’s involves 
full utilization of datalink weather services, ATIS, and ATC clearances/amendments. The 
initial implementation of this is the pre-departure clearance program that we are now 
utilizing at ORD, DFW, and SFO. 

The purpose of the attached survey is to obtain the pilot’s perspective on whether or 
not certain communication elements in the current ATC environment are significant I feel 
that it is important to get our input anytime the implementation of a new technology has the 
potential of changing an existing mode of operation as much as this does. We, as pilots, 
will be the ones who have to work in the environments that others implement so it is 
important for us to take advantage of any opportunity we can to influence the design of 
future systems. Researchers value the opinions of pilots concerning new systems but in 
the past it has been difficult to get large scale input due to the nature of surveying. 

The results of the survey will be used in my thesis which in turn will be utilized by 
NASA- Ames human factors group to help in determining guidelines for datalink 
implementation. It should be noted that this study is being funded by a grant from NASA- 
Ames and is being carried out by the Aeronautical Systems Lab at MIT. The research is 
completely independent of AAL and the APA although both Carl Price of AAL and Mike 
Shanholtz of the APA are aware of and support the study. If you find the time to complete 
this survey, please return it to the marked box above my ORD mailbox. 

Thanks, 

Alan H. Midkiff 
AA#13360 
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SURVEY ON 

DATALINK ATC MESSAGE EXCHANGE 


The Department of Aeronautics and Astronautics at the Massachusetts Institute of 
Technology is currently evaluating ATC message exchange using digital datalink. ATC 
communications via the datalink would be displayed either textually or graphically in the 
cockpit thus minimizing the need for voice communications and relieving congested radio 
frequencies. 

Among the many facets being studied in this research is the significance of “party line” 
information, which pilots acquire by overhearing messages intended for other aircraft. In 
the datalink environment, the availability of such information may be minimal due to the 
reduction of voice communications. The purpose of this survey is to obtain a general 
outlook on the presence of party line information to pilots relative to the environment in 
which they are operating. 

In the survey you are asked to rank the importance, availability, and accuracy of the 
information derived from party line for various phases of a typical flight, and for a few 
miscellaneous items. In addition, there is a free comment area, a few datalink questions, 
and a background section. 

Participation in this survey is completely voluntary. It is not necessary to give your name 
at any point. You may decline to answer any of the questions in this survey, without 
prejudice. All surveys will be de-identified and all information obtained from any 
individual survey will be kept confidential by the researchers at MIT. 


For further information about this study, please feel free to contact: 


Principal Investigator: 


Research Assistant: 


R. John Hansman, Jr., PhD 

Boeing Associate Professor of Aeronautics 

77 Massachusetts Ave. 

Rm. 33-115 
Cambridge, MA 02139 
(617) 253-2271 


Alan H. Midkiff 
Aeronautical Systems 
Laboratory 

77 Massachusetts Ave. 
Rm. 37-438 
Cambridge, MA 02139 
(617) 253-0993 


Thank you for your time and cooperation. 
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PARTY LINE INFORMATION 
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PARTY LINE INFORMATION (cont.) 
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MISCELLANEOUS PARTY LINE INFORMATION 
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ADDITIONAL PARTY LINE QUESTIONS 


Considering the advantages of datalink (i.e. quiet frequency, discrete aircraft addressing, frequency congestion 
relief, unambiguous clearances, etc.) and of party line information (situational/environmental awareness, 
traffic/ride information, etc.), would you tend to support a datalink environment or the current environment 
containing party line information (PLI)? 


Choose one: 

1 

2 

3 

4 5 

DATALINK 


EQUAL 

VHF VOICE 

ONLY 


DISTRIBUTION 

COMMUNICATION 



VOICE / DATALINK 

ONLY (PLI) 


If some mechanism could be developed to datalink critical PLI to the aircraft (eg. a status display with 
current wx, sequencing, and/or holding information), would you tend to support a datalink environment or 
the present PLI environment? 


Choose one: 

1 

2 

3 

4 5 

DATALINK 


EQUAL 

VHF VOICE 

ONLY 


DISTRIBUTION 

COMMUNICATION 



VOICE / DATALINK 

ONLY (PLI) 


Additional comments (use the back of this page if necessary): 


Are you TC AS qualified? YES NO If so, please comment on the effectiveness of TCAS as a 

compensational device for situational awareness, sequencing, and traffic watch in the datalink environment. 


Please provide any ideas you might have or like to see concerning datalinking PLI to the aircraft. 


Enter any comments about the significance of party line information, e.g., what benefits and/or problems 
do you think you encounter that were not included on the previous pages (use back if necessary). 
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BACKGROUND INFORMATION 


How long have you been employed as a professional pilot? yrs 

What is your total number of flying hours? 

Your flying background is primarily military non-military (choose one). 

What is your age? 

How long have you been flying with American Airlines? yrs mos 


Please estimate all flight hours in your current and previous aircraft (including your experience with other 
airlines and/or military flying) and indicate your crew position on that aircraft. 


Current 

TvDe 

Hrs. in Seat 

CAPT 

F/O 

S/O 

Immediat prior 

* J r 

Tvoe 

Hrs. in Seat 

CAPT 

F/O 

S/O 

Other 

Tvoe 

Hrs. in Seat 

CAPT 

F/O 

S/O 

Other 

/ r 

Tvoe 

Hrs. in Seat 

CAPT 

F/O 

S/O 

Other 

* j K 

Tvoe 

Hrs. in Seat 

CAPT 

F/O 

S/O 

Other 

J 

Tvoe 

Hrs. in Seat 

CAPT _ 

F/O ' 

S/O 

Other 

* J r 

Tvoe 

Hrs. in Seat 

CAPT 

F/O 

S/O 

Other 

/ r 

Type 

Hrs. in Seat 

CAPT _ 

F/O 

S/O 


Do you use a personal computer? YES NO 

How satisfied are you with predeparture clearances through the ACARS datalink? Include any additional 
comments. 



Appendix C: 

Survey Results Tabulation 
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Ground 

Ops 


Departure 


Cruise 


Descent 


Terminal 

Area 


Final 

Approach 


PARTY LINE SURVEY RESULTS (N=184) 


IMPORTANCE 

Mean Standard Deviation 


a/c crossing active rwy 

4.65 

a/c crossing activ rwy 

0.69 

other a/c "hold short" 

4.06 

ground sequencing 

1.04 

controller errors 

4.01 

other a/c "hold short" 

1.09 

routing to rwy 

3.45 

controller errors 

1.10 

ground sequencing 

3.27 

next comm freq 

1.13 

next comm freq 

2.60 

routing to rwy 

1.15 

wx situation 

4.40 

wx situation 

0.70 

ride reports 

4.06 

ride reports 

0.84 

traffic watch 

3.92 

traffic watch 

1.07 

controller errors 

3.91 

controller errors 

1.10 

next comm freq 

2.90 

next comm freq 

1.14 

wx situation 

4.38 

wx situation 

0.68 

ride reports 

4.10 

ride reports 

0.76 

controller errors 

3.63 

winds aloft 

0.98 

traffic watch 

3.36 

sequencing 

1.04 

sequencing 

3.04 

next comm freq 

1.08 

winds aloft 

2.85 

controller errors 

1.17 

next comm freq 

2.70 

traffic watch 

1.17 

wx situation 

4.41 

wx situation 

0.63 

hold situation/EFC 

4.08 

ride reports 

0.82 

traffic watch 

3.98 

hold situation/EFC 

0.88 

ride reports 

3.97 

sequencing 

0.93 

controller errors 

3.82 

traffic watch 

1.01 

sequencing 

3.48 

next comm freq 

1.07 

next comm freq 

3.08 

controller errors 

1.11 

wx situation 

4.47 

wx situation 

0.61 

traffic watch 

4.22 

terminal routing 

0.81 

approach clearance 

4.16 

sequencing 

0.85 

terminal routing 

4.09 

traffic watch 

0.91 

hold situation/EFC 

4.01 

approach clearance 

0.91 

controller errors 

3.97 

hold situation/EFC 

0.91 

ride reports 

3.86 

ride reports 

0.92 

sequencing 

3.82 

controller errors 

1.13 

next comm freq 

3.46 

next comm freq 

1.15 

windshear 

4.87 

windshear 

0.34 

a/c on your ldg rwy 

4.81 

a/c on your ldg rwy 

0.49 

braking action 

4.63 

braking action 

0.53 

missed approach-wx 

4.61 

missed approach-wx 

0.69 

wx situation 

4.50 

wx situation 

0.83 

go around 

4.32 

go around 

0.85 

traffic watch 

4.09 

traffic watch 

1.05 

sequencing 

3.71 

sequencing 

1.06 

taxiway turnoff 

3.46 

taxiway turnoff 

1.17 

next comm freq 

3.26 

next comm freq 

1.29 
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ACCURACY 


Ground 

Ops 


Departure 


Cruise 


Descent 


Terminal 

Area 


Final 

Approach 


Mean 


a/c crossing active rwy 

4.12 

next comm freq 

3.85 

other a/c "hold short" 

3.81 

routing to rwy 

3.64 

ground sequencing 

3.50 

controller errors 

3.26 

ride reports 

3.74 

wx situation 

3.71 

next comm freq 

3.63 

traffic watch 

3.40 

controller errors 

3.00 

wx situation 

3.83 

ride reports 

3.81 

next comm freq 

3.77 

traffic watch 

3.45 

winds aloft 

3.42 

sequencing 

3.30 

controller errors 

2.97 

wx situation 

3.89 

ride reports 

3.78 

next comm freq 

3.77 

hold situation/EFC 

3.67 

sequencing 

3.46 

traffic watch 

3.44 

controller errors 

3.06 

approach clearance 

4.09 

next comm freq 

4.00 

wx situation 

3.91 

terminal routing 

3.84 

ride reports 

3.73 

hold situation/EFC 

3.66 

traffic watch 

3.63 

sequencing 

3.62 

controller errors 

3.15 

a/c on your ldg rwy 

4.12 

next comm freq 

4.09 

wx situation 

4.07 

missed approach-wx 

4.06 

go around 

4.02 

braking action 

3.91 

windshear 

3.84 

traffic watch 

3.79 

sequencing 

3.79 

taxiway turnoff 

3.38 


Standard Deviation 
a/c crossing activ rwy 0.88 


other a/c "hold short" 

0.90 

next comm freq 

0.97 

controller errors 

0.97 

routing to rwy 

0.99 

ground sequencing 

0.99 

ride reports 

0.79 

wx situation 

0.82 

traffic watch 

0.95 

controller errors 

1.02 

next comm freq 

1.09 

ride reports 

0.77 

wx situation 

0.81 

traffic watch 

0.91 

winds aloft 

0.95 

next comm freq 

1.00 

controller errors 

1.02 

sequencing 

1.04 

ride reports 

0.75 

wx situation 

0.76 

sequencing 

0.83 

hold situation/EFC 

0.85 

traffic watch 

0.86 

controller errors 

0.94 

next comm freq 

0.98 

terminal routing 

0.77 

wx situation 

0.78 

approach clearance 

0.79 

ride reports 

0.83 

sequencing 

0.84 

hold situation/EFC 

0.86 

traffic watch 

0.87 

next comm freq 

0.91 

controller errors 

0.98 

a/c on your ldg rwy 

0.77 

braking action 

0.81 

wx situation 

0.83 

go around 

0.83 

missed approach-wx 

0.86 

sequencing 

0.87 

traffic watch 

0.88 

windshear 

0.89 

next comm freq 

0.92 

taxiway turnoff 

0.99 
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AVAILABILITY 


Ground 

Ops 


Departure 


Cruise 


Descent 


Terminal 

Area 


Final 

Approach 


Mean 

3.86 

a/c crossing active rwy 
other a/c "hold short" 

3.82 

routing to rwy 

3.81 

next comm freq 

3.75 

ground sequencing 

3.40 

controller errors 

2.96 

ride reports 

3.86 

wx situation 

3.70 

next comm freq 

3.69 

traffic watch 

3.44 

controller errors 

2.81 

ride reports 

3.99 

wx situation 

3.86 

next comm freq 

3.76 

traffic watch 

3.24 

sequencing 

2.99 

winds aloft 

2.79 

controller errors 

2.74 

wx situation 

3.83 

hold situation/EFC 

3.81 

next comm freq 

3.79 

ride reports 

3.77 

traffic watch 

3.44 

sequencing 

3.40 

controller errors 

2.94 

approach clearance 

4.07 

next comm freq 

3.96 

wx situation 

3.94 

terminal routing 

3.86 

hold situation/EFC 

3.77 

traffic watch 

3.69 

ride reports 

3.68 

sequencing 

3.48 

controller errors 

2.99 

wx situation 

4.00 

next comm freq 

3.98 

a/c on your ldg rwy 

3.91 

braking action 

3.90 

windshear 

3.89 

missed approach-wx 

3.88 

go around 

3.83 

traffic watch 

3.67 

sequencing 

3.64 

taxiway turnoff 

3.25 


Standard Deviation 


other a/c "hold short" 

0.94 

routing to rwy 

0.99 

controller errors 

1.00 

ground sequencing 

1.05 

a/c crossing activ rwy 

1.10 

next comm freq 

1.15 

ride reports 

0.86 

wx situation 

0.90 

traffic watch 

0.95 

next comm freq 

1.03 

controller errors 

1.06 

ride reports 

0.80 

wx situation 

0.84 

winds aloft 

0.94 

traffic watch 

0.95 

controller errors 

0.96 

next comm freq 

0.98 

sequencing 

1.05 

wx situation 

0.83 

hold situation/EFC 

0.86 

ride reports 

0.86 

sequencing 

0.91 

next comm freq 

0.93 

traffic watch 

0.96 

controller errors 

0.97 

wx situation 

0.79 

approach clearance 

0.82 

sequencing 

0.87 

next comm freq 

0.88 

terminal routing 

0.90 

traffic watch 

0.90 

ride reports 

0.91 

hold situation/EFC 

0.92 

controller errors 

1.00 

wx situation 

0.91 

sequencing 

0.91 

braking action 

0.92 

traffic watch 

0.94 

windshear 

0.95 

missed approach-wx 

0.95 

go around 

0.95 

a/c on your ldg rwy 

0.96 

next comm freq 

0.97 

taxiway turnoff 

1.03 
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BY PHASE 


Importance 


Availability 


Accuracy 


Importance 


Availability 


Accuracy 


Mgan 


Standard Deviation 


final approach 
terminal area 
departure 
descent 
ground ops 
cruise 

final approach 
terminal area 
ground ops 
descent 
departure 
cruise 

final approach 
terminal area 
ground ops 
descent 
cruise 
departure 


4.23 

terminal area 

0.28 

4.01 

descent 

0.43 

3.84 

departure 

0.56 

3.83 

final approach 

0.57 

3.67 

cruise 

0.63 

3.44 

ground ops 

0.72 

3.79 

final approach 

0.23 

3.71 

terminal area 

0.33 

3.60 

descent 

0.33 

3.57 

ground ops 

0.36 

3.50 

departure 

0.41 

3.34 

cruise 

0.53 

3.91 

final approach 

0.22 

3.74 

terminal area 

0.28 

3.70 

descent 

0.28 

3.58 

ground ops 

0.30 

3.51 

departure 

0.31 

3.50 

cruise 

0.32 


ACROSS ALL PHASES 


Mean 


wx situation 

4.43 

hold situation/EFC 

4.04 

ride reports 

4.00 

controller errors 

3.87 

traffic watch 

3.87 

sequencing 

3.51 

next comm freq 

3.00 

wx situation 

3.86 

ride reports 

3.83 

next comm freq 

3.82 

hold situation/EFC 

3.79 

traffic watch 

3.50 

sequencing 

3.38 

controller errors 

2.89 

wx situation 

3.88 

next comm freq 

3.85 

ride reports 

3.76 

hold situation/EFC 

3.67 

sequencing 

3.54 

traffic watch 

3.48 

controller errors 

3.09 


Standard Deviation 


wx situation 

0.05 

hold situation/EFC 

0.05 

ride reports 

0.10 

controller errors 

0.15 

next comm freq 

0.33 

sequencing 

0.35 

traffic watch 

0.36 

hold situation/EFC 

0.03 

controller errors 

0.10 

wx situation 

0.11 

next comm freq 

0.12 

ride reports 

0.14 

traffic watch 

0.19 

sequencing 

0.27 

hold situation/EFC 

0.00 

ride reports 

0.04 

traffic watch 

0.10 

controller errors 

0.12 

wx situation 

0.13 

next comm freq 

0.17 

sequencing 

0.21 
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MISCELLANEOUS 


Importance 


Availability 


Accuracy 


Msan 


call sign confusion 4.53 

controller urgency 4.22 

ATC prob/lost comm 4.22 

navaid problems 3.96 

other pilot urgency 3.83 

sector congestion 3.70 

controller exp level 3.43 

bkgnd reassurance 3.31 

other pilot exp level 3.08 


controller urgency 3.93 

sector congestion 3.89 

bkgnd reassurance 3.78 

call sign confusion 3.7 1 

other pilot urgency 3.60 

controller exp level 3.49 

ATC prob/lost comm 3.19 

navaid problems 3.19 

other pilot exp level 3.14 


controller urgency 3.73 

bkgnd reassurance 3.71 

sector congestion 3.61 

navaid problems 3.59 

call sign confusion 3.51 

other pilot urgency 3.49 

ATC prob/lost comm 3.40 

controller exp level 3.06 

other pilot exp level 2.78 


Standard Deviation 


call sign confusion 0.74 
controller urgency 0.85 
navaid problems 0.89 
ATC prob/lost comm 0.93 
other pilot urgency 1 .03 
sector congestion 1.03 
controller exp level 1.04 
other pilot exp level 1.10 
bkgnd reassurance 1.16 


call sign confusion 0.94 
sector congestion 0.98 
controller urgency 0.99 
navaid problems 1.02 
ATC prob/lost comm 1 .04 
bkgnd reassurance 1 .07 
other pilot urgency 1 .07 
controller exp level 1.08 
other pilot exp level 1.16 


sector congestion 0.89 

ATC prob/lost comm 0.96 

navaid problems 0.98 

controller urgency 1.02 

other pilot exp level 1.02 

other pilot urgency 1.03 

controller exp level 1.03 

call sign confusion 1 .04 

bkgnd reassurance 1.18 



MISCELLANEOUS/BACKGROUND 


Deviation ( 

DL-PLI 

DL-PLI with compensation 

SHIFT (towards "DATALINK ONLY") 

TCAS ( 1 unfavorable. . ,3=favorable) 
PDC (l=unfavorable...3=favorable) 

AGE 
AA years 
Total years 

Flight experience (total hours) 


Mean 

Standard 

3.03 

0.82 

2.71 

0.92 

0.50 

0.81 

2.18 

0.76 

2.83 

0.48 

40.8 

8.13 

10.1 

8.02 

17.0 

8.03 

8914 

5027 


Total Number 


Captains 

86 

First officers 

77 

Second officers 

9 

Military background 

88 

Non-military background 

89 

TCAS qualified 

111 

EFIS qualified 

33 

PC user 

111 



Appendix D: 


Statistical Analysis of Survey Results 
(95% Confidence Interval Analysis) 


184 pilots responded to the survey. The mean response ratings for each survey element are listed 
in the following pages. The means are given with a 95% confidence interval assuming a normal 
distribution of data. The standard deviations are also presented to give an indication of variability 
in individual responses. 
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PARTY LINE SURVEY RESULTS (n=184) 
IMPORTANCE 


Ground 

Ops 


Departure 


Cruise 


Descent 


Terminal 

Area 


Final 

Approach 



Mean (ft ± 1.96o/Vn) 

Stdev a 

a/c crossing active rwy 

4.65 ± .10 

0.69 

other a/c "hold short" 

4.06 ± .16 

1.09 

controller errors 

4.01 ± .16 

1.10 

routing to rwy 

3.45 ± .17 

1.15 

ground sequencing 

3.27 ± .15 

1.04 

next comm freq 

2.60 ±.16 

1.13 

wx situation 

4.40 ± .10 

0.70 

ride reports 

4.06 ± .12 

0.84 

traffic watch 

3.92 ±.15 

1.07 

controller errors 

3.91 ±.16 

1.10 

next comm freq 

2.90 ±.16 

1.14 

wx situation 

4.38 ±.10 

0.68 

ride reports 

4.10± .11 

0.76 

controller errors 

3.63 ±.17 

1.17 

traffic watch 

3.36 ±.17 

1.17 

sequencing 

3.04 ± .15 

1.04 

winds aloft 

2.85 ± .14 

0.98 

next comm freq 

2.70 ± .16 

1.08 

wx situation 

4.41 ± .09 

0.63 

hold situation/EFC 

4.08 ±.13 

0.88 

traffic watch 

3.98 ± .15 

1.01 

ride reports 

3.97 ± .12 

0.82 

controller errors 

3.82 ± .16 

1.11 

sequencing 

3.48 ±.13 

0.93 

next comm freq 

3.08 ±.15 

1.07 

wx situation 

4.47 ± .09 

0.61 

traffic watch 

4.22 ± .13 

0.91 

approach clearance 

4.16 ± .13 

0.91 

terminal routing 

4.09 ± .12 

0.81 

hold situation/EFC 

4.01 ± .13 

0.91 

controller errors 

3.97 ± .16 

1.13 

ride reports 

3.86 ±.13 

0.92 

sequencing 

3.82 ± .12 

0.85 

next comm freq 

3.46 ± .17 

1.15 

windshear 

4.87 ± .05 

0.34 

a/c on your ldg rwy 

4.81 ±.07 

0.49 

braking action 

4.63 ± .08 

0.53 

missed approach - wx 

4.61 ±.10 

0.69 

wx situation 

4.50 ±.12 

0.83 

go around 

4.32 ±.12 

0.85 

traffic watch 

4.09 ±.15 

1.05 

sequencing 

3.71 ±.15 

1.06 

taxiway turnoff 

3.46 ±.17 

1.17 

next comm freq 

3.26 ±.19 

1.29 
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AVAILABILITY 


Ground 

Ops 


Departure 


Cruise 


Descent 


Terminal 

Area 


Final 

Approach 



Mean Cu ± 1.96 a / Vn ) 

Stdev a 

a/c crossing active rwy 

3.86 ± .16 

1.10 

other a/c "hold short" 

3.82 ± .14 

0.94 

routing to rwy 

3.81 ± .14 

0.99 

next comm freq 

3.75 ± .17 

1.15 

ground sequencing 

3.40 ± .15 

1.05 

controller errors 

2.96 ±.14 

1.00 

ride reports 

3.86 ±.12 

0.86 

wx situation 

3.70 ±.13 

0.90 

next comm freq 

3.69 ± .15 

1.03 

traffic watch 

3.44 ± .14 

0.95 

controller errors 

2.81 ±.15 

1.06 

ride reports 

3.99 ± .12 

0.80 

wx situation 

3.86 ± .12 

0.84 

next comm freq 

3.76 ± .14 

0.98 

traffic watch 

3.24 ± .14 

0.95 

sequencing 

2.99 ± .15 

1.05 

winds aloft 

2.79 ± .14 

0.94 

controller errors 

2.74 ± .14 

0.96 

wx situation 

3.83 ± .12 

0.83 

hold situation/EFC 

3.81 ± .12 

0.86 

next comm freq 

3.79 ± .13 

0.93 

ride reports 

3.77 ±.12 

0.86 

traffic watch 

3.44 ± .14 

0.96 

sequencing 

3.40 ± .13 

0.91 

controller errors 

2.94 ± .14 

0.97 

approach clearance 

4.07 ± .12 

0.82 

next comm freq 

3.96 ± .13 

0.88 

wx situation 

3.94 ± .11 

0.79 

terminal routing 

3.86 ± .13 

0.90 

hold situation/EFC 

3.77 ± .13 

0.92 

traffic watch 

3.69 ± .13 

0.90 

ride reports 

3.68 ± .13 

0.91 

sequencing 

3.48 ± .13 

0.87 

controller eiTors 

2.99 ±.14 

1.00 

wx situation 

4.00 ± .13 

0.91 

next comm freq 

3.98 ±.14 

0.97 

a/c on your ldg rwy 

3.91 ± .14 

0.96 

braking action 

3.90 ± .13 

0.92 

windshear 

3.89 ± .14 

0.95 

missed approach - wx 

3.88 ± .14 

0.95 

go around 

3.83 ±.14 

0.95 

traffic watch 

3.67 ± .14 

0.94 

sequencing 

3.64 ± .13 

0.91 

taxiway turnoff 

3.25 ±.15 

1.03 
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ACCURACY 


Ground 

Ops 


Departure 


Cruise 


Descent 


Terminal 

Area 


Final 

Approach 



Mean (u ± 1.96a/Vn) 

Stdev a 

a/c crossing active rwy 

4.12 ± .13 

0.88 

next comm freq 

3.851.14 

0.97 

other a/c "hold short" 

3.81 1.13 

0.90 

routing to rwy 

3.641.14 

0.99 

ground sequencing 

3.501.14 

0.99 

controller errors 

3.261 .14 

0.97 

ride reports 

3.741 .11 

0.79 

wx situation 

3.71 1 .12 

0.82 

next comm ffeq 

3.631.16 

1.09 

traffic watch 

3.401 .14 

0.95 

controller errors 

3.001 .15 

1.02 

wx situation 

3.831 .12 

0.81 

ride reports 

3.81 1 .11 

0.77 

next comm freq 

3.771 .14 

1.00 

traffic watch 

3.45 1 .13 

0.91 

winds aloft 

3.421 .14 

0.95 

sequencing 

3.301 .15 

1.04 

controller errors 

2.971.15 

1.02 

wx situation 

3.891 .11 

0.76 

ride reports 

3.781 .11 

0.75 

next comm freq 

3.77 1 .14 

0.98 

hold situation/EFC 

3.671.12 

0.85 

sequencing 

3.461 .12 

0.83 

traffic watch 

3.441.12 

0.86 

controller errors 

3.061 .14 

0.94 

approach clearance 

4.09 1 .11 

0.79 

next comm ffeq 

4.00 1 .13 

0.91 

wx situation 

3.91 1 .11 

0.78 

terminal routing 

3.841 .11 

0.77 

ride reports 

3.731 .12 

0.83 

hold situation/EFC 

3.661 .12 

0.86 

traffic watch 

3.631.13 

0.87 

sequencing 

3.621 .12 

0.84 

controller errors 

3.151.14 

0.98 


a/c on your ldg rwy 
next comm freq 
wx situation 
missed approach - wx 
go around 
braking action 
windshear 
traffic watch 
sequencing 
taxiway turnoff 


4.121 .11 

0.77 

4.091 .13 

0.92 

4.07 1 .12 

0.83 

4.061 .12 

0.86 

4.021 .12 

0.83 

3.91 1 .12 

0.81 

3.841 .13 

0.89 

3.791 .13 

0.88 

3.791 .13 

0.87 

3.381.14 

0.99 
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BY PHASE 


Importance 


Availability 


Accuracy 


Importance 


Availability 


Accuracy 



Mean (|i.± 1.96a/Vn) 

Stdev o 

final approach 

4.23 ± .08 

0.57 

terminal area 

4.01 ± .04 

0.28 

departure 

3.84 ± .08 

0.56 

descent 

3.83 ± .06 

0.43 

ground ops 

3.67 ± .10 

0.72 

cruise 

3.44 ± .09 

0.63 

final approach 

3.79 ± .03 

0.23 

terminal area 

3.71 ± .05 

0.33 

ground ops 

3.60 ± .05 

0.36 

descent 

3.57 ± .05 

0.33 

departure 

3.50 ± .06 

0.41 

cruise 

3.34 ± .08 

0.53 

final approach 

3.91 ± .03 

0.22 

terminal area 

3.74 ± .04 

0.28 

ground ops 

3.70 ± .04 

0.30 

descent 

3.58 ± .04 

0.28 

cruise 

3.51 ± .05 

0.32 

departure 

3.50 ± .04 

0.31 


ACROSS ALL PHASES 



Mean ffx ± 1.96a/Vn) 

Stdev a 

wx situation 

4.43 

± 

.01 

0.05 

hold situation/EFC 

4.04 

± 

.01 

0.05 

ride reports 

4.00 

± 

.01 

0.10 

controller errors 

3.87 

± 

.02 

0.15 

traffic watch 

3.87 

+ 

.05 

0.36 

sequencing 

3.51 

+ 

.05 

0.35 

next comm freq 

3.00 

± 

.05 

0.33 

wx situation 

3.86 

± 

.02 

0.11 

ride reports 

3.83 

± 

.02 

0.14 

next comm freq 

3.82 

± 

.02 

0.12 

hold situation/EFC 

3.79 

± 

.00 

0.03 

traffic watch 

3.50 

± 

.03 

0.19 

sequencing 

3.38 

± 

.04 

0.27 

controller errors 

2.89 

± 

.01 

0.10 

wx situation 

3.88 

± 

.02 

0.13 

next comm freq 

3.85 

± 

.02 

0.17 

ride reports 

3.76 

± 

.01 

0.04 

hold situation/EFC 

3.67 

± 

.00 

0.00 

sequencing 

3.54 

± 

.03 

0.21 

traffic watch 

3.48 

± 

.01 

0.10 

controller errors 

3.09 

± 

.02 

0.12 



MISCELLANEOUS 


Importance 


Availability 


Accuracy 



Mean ± 1.96o/Vn) 

Stdev q 

call sign confusion 

4.53 ±.11 

0.74 

controller urgency 

4.22 ±.12 

0.85 

ATC prob/lost comm 

4.22 ±.13 

0.93 

navaid problems 

3.96 ± .13 

0.89 

other pilot urgency 

3.83 ±.15 

1.03 

sector congestion 

3.70 ±.15 

1.03 

controller exp level 

3.43 ±.15 

1.04 

bkgnd reassurance 

3.31 ±.17 

1.16 

other pilot exp level 

3.08 ±.16 

1.10 

controller urgency 

3.93 ±.14 

0.99 

sector congestion 

3.89 ± .14 

0.98 

bkgnd reassurance 

3.78 ±.15 

1.07 

call sign confusion 

3.71 ± .14 

0.94 

other pilot urgency 

3.60 ± .15 

1.07 

controller exp level 

3.49 ± .16 

1.08 

ATC prob/lost comm 

3.19 ±.15 

1.04 

navaid problems 

3.19 ± .15 

1.02 

other pilot exp level 

3.14 ± .17 

1.16 

controller urgency 

3.73 ±.15 

1.02 

bkgnd reassurance 

3.71 ±.17 

1.18 

sector congestion 

3.61 ±.13 

0.89 

navaid problems 

3.59 ±.14 

0.98 

call sign confusion 

3.51 ± .15 

1.04 

other pilot urgency 

3.49 ± .15 

1.03 

ATC prob/lost comm 

3.40 ± .14 

0.96 

controller exp level 

3.06 ± .15 

1.03 

other pilot exp level 

2.78 ± .15 

1.02 



MISCELLANEOUS/BACKGROUND 


DL-PLI 

DL-PLI with compensation 

SHIFT (towards "DATALINK ONLY") 

TCAS (l=unfavorable...3=favorable) 
PDC (l=unfavorable...3=favorable) 

AGE 
AA years 
Total years 

Flight experience (total hours) 


Captains 
First officers 
Second officers 

Military background 
Non-military background 

TCAS qualified 
EFIS qualified 
PC user 


Mean (|ii ± 1.96o/Vn) 

Stdeva 

3.03 ±.12 

0.82 

2.71 ± .13 

0.92 

0.50 +.12 

0.81 

2.18 ±.11 

0.76 

2.83 ± .07 

0.48 

40.8 

8.13 

10.1 

8.02 

17.0 

8.03 

8914 

5027 

Total Number 



86 

77 

9 


88 

89 

111 

33 

111 



Appendix E: 


Full-Mission Simulation Script 
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H 


= |£3 


ADVANCED CONCEPTS FLIGHT SIMULATOR 

Data Link/Party Line 
Air Traffic Control 

PHRASEOLOGY 

Zulu +8 


Los Angeles to San Francisco: 


ATIS (LAX DEPARTURE) 


1 

This is the Los Angeles airport information XRAY 0045Z. Weather measured ceiling 1000 
overcast, visibility 1 haze and smoke. Temperature 57 dewpoint 40 wind 260 at 10 
altimeter 3002. Traffic landing and departing runways 24 and 25. Advise on initial contact 
you have Xray. 

VENTURA 1 DEPARTURE, DIRECT VENTURA, DIRECT SAN MARCUS, DIRECT 
BIG SUR, DIRECT SAN FRANCISCO 

After ACFS calls 

LAX CLR: XXX is cleared as filed via the Ventura 1 , maintain 5,000 expect flight 
level 310 three minutes after departure, departure control frequency will be 125.2 
squawk 3647 

After ACFS calls 


1L 

LAX GND: XXX taxi runway 25L via Juliet, hold short of 25R 


GROUND EVENT #1: Subject will encounter crossing traffic (UAL450)on taxiway. 
Ground control will make repeated calls to UAL450 telling him to hold for ACFS. UAL450 
does not respond. Ground control will stop ACFS if necessary 

LAX GND: United 450, hold short of the next taxiway for the United aircraft. 

REPEAT ABOVE CALL SEVERAL TIMES 


LAX GND: XXX, cross 25R contact the tower on 1 33.9 

After ACFS calls 

LAX TWR: King Air 56M hold short of 25L 
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N56M: roger, holding 


LAX TWR: XXX, taxi into position and hold 25L 

GROUND EVENT #2: A fter being cleared for takeoff, subject will hear the I 
tower clear another aircraft to cross the runway farther downfield. I 

1 

LAX TWR: XXX, wind 260 at 10, cleared for takeoff 
LAX TWR: King Air 56M cross 25L, contact ground point 75 
N56M: roger, 56 M 

when ACTS leaves 500’ 

1 

LAX TWR: XXX contact departure control on 125.2 

After ACFS calls 

& 

LAX DEP: XXX, departure control radar contact, climb and maintain 10,000 

when ACFS is 20 miles west of LAX 

2 . 

LAX DEP: XXX, turn right heading 300, cleared direct Ventura, contact Los Angeles center 
on 135.5 

After ACFS calls 

M 

1 ,AX CTR: XXX roger, climb and maintain flight level 230, expedite through flight level 
180 

LAX CTR: TWA 127 climb and maintain flight level 280 
TWA 1 27 : roger up to 280, were out of nnn 
LAX CTR: TWA 127 contact the center on 135.3 
TWA 127: 135.3, so long 

after ACFS climbs out of flight level 200 

M 

LAX CTR: XXX, climb and maintain flight level 280, contact the center on 1 35.3 

when ACFS approaches MQQ 
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12 

LAX CTR: XXX, contact Oakland center on 134.5 

After ACFS calls 

OAK CTR: XXX Oakland center, roger 


ENROUTE EVENT: Approaching Zonal, subject will hear preceding aircraft 
give an unfavorable ride report (light to moderate turbulence) at the same 
altitude as subject. _____ 


TWA 1 27: Ah center this is TWA 127, we’re getting bounced around here pretty good 
northwest of Zonal, we'd like to reduce to 280 kts. 

OAK CTR: TWA 127 roger 


When ACFS is at ZONAL 

11 

OAK CTR: XXX, turn left heading 240 for spacing, maintain flight level 280, do not exceed 
260 kts 


when ACFS is 10 miles west of course 

M 

OAK CTR: XXX turn right heading 295 


after 5 miles 


M 

OAK CTR: XXX, intercept J501, resume your own navigation (this is a mm inter cent 

If they (jo no! question the clearance 111 5 miles ask it they are go ing IQ SW7 lM Udm 
nr Luve them the tight, [uni to 350) 

when ACFS is 30 south of B i g Sur 

11 

OAK CTR: XXX, cleared to the San Francisco airport via the Big Sur arrival, contact the 
center on 128.7 


After ACFS calls 

. 11 

OAK CTR: XXX roger, San Francisco altimeter is 30.04 


m 

OAK CTR: XXX traffic 1 1 o'clock 10 miles crossing left to right above you 
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12 

OAK CTR: XXX traffic 1 o'clock 8 miles southeastbound. a Lear Jet below you. 
ATIS (SFO ARRIVAL) 

M 

This is San Francisco airport information Delta 0145Z. Weather measured ceiling 300 overcast 
visibility 5. Temperature 50 dewpoint 41 wind light and variable altimeter 3004. Traffic landing 
runways 28 departing runways 1 . Advise on initial contact you have Delta. 

ARRIVAL DEVIATION EVENT: Subject is given an off route vector after I 
BSR and a subsequent re-intercept. I 


When ACFS passes CAR MI: 

11 

OAK CTR: XXX, turn left heading 270 vector for spacing maintain 16,000 


OAK CTR: XXX traffic 12 o'clock 10 miles opposite direction below’ you 

when ACFS is 15 miles west of course 

11 

OAK CTR: XXX, turn right heading 350 intercept the Oakland 151 radial and resume the 
Big Sur arrival, contact Bay approach on 133.9 

After ACFS calls 

BAY APP: XXX roger descend and maintain 6,000 

11 

BAY APP: XXX traffic 11 o'clock 10 miles eastbound above you 
BAY APP: Mexicana 1248, advise prior to reducing below 250 

M 

BAY APP: XXX traffic 1 o'clock 10 miles southbound above you 

11 

BAY APP: XXX reduce to 210 knots contact Bay on 135.6 

After ACFS calls 

BAY APP: Mexicana 1248 your 10 miles from BRIJJ, turn left heading 310, maintain 2,000 
until established on the localizer, cleared for ILS 28 right approach 

M 

BAY APP: XXX traffic 1 1 o'clock 5 miles east bound restricted below you 
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22 

BAY APP: XXX turn right heading 350, descend and maintain 4,000, reduce to 180 knots 

MOTE. 

There are two other aircraft inbound to the airport - one from over CEDES, the other 
downwind north from over PYE 

when ACFS is about 15 miles from airport 


M 

BAY APP: XXX is 10 miles from BRTJJ, turn left heading 300, maintain 2,000 until 
established on the localizer, cleared for ILS 28 right approach, maintain 160 knots until 
BRIJJ 


SE5 ARRIVAL EVENT: Subject is following a non-USA air carrier to runway 28R 
who is difficult to understand. This aircraft rolls out on the runway and is confused about 
which turnoff to take. After finally turning off the runway his landing gear collapses 
closing the airport. If the ACFS crew does not go around by 200 feet the tower will issue a 
go around. They are given a clearance to CROIT intersection to hold. Subject will 
eventually call company and will be sent to SMF. 


when ACFS is 10 miles from airport 

M 

BAY APP: XXX contact the tower now 

After ACFS calls 

M 

SFO TWR: XXX roger, reduce to approach speed, cleared to land 28 right 
SFO TWR: Mexicana 1248 wind light and variable cleared to land 28 right 
MEX1248: roger 


Note: 

The tower will make repeated calls to MEX1248 when he doesn't get off the runway - the 
pilot is confused as to which exit to take. Issue a go around to the ACFS ONLY, if they 
don't go around on their own by 200 feet 


SFO TWR: XXX go around, emergency in progress, maintain runway heading, climb and 
maintain 4,000 contact Bay Departure on 120.9 

When ACFS is 5 mile s west of SFO 

M 

BAY DEP: XXX roger, turn right heading 350, climb and maintain 5,000 
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as 

BAY DEP: XXX on that heading intercept the SAU 035 radial cleared to CROIT 
intersection. Expect holding at CROI T 

DAL427: Bay this is DAL427 checking in at 8,000 going to CROIT for holding 
BAY DEP: DAL427 roger expect furtlier cleai'ance in 30 minutes 
DAL427: roger 

WWM1398: Bay this is Wings Westl398 at 9,000 about to enter holding at CROIT 
BAY DEP: Wings West 1398 roger, expect further clearance in 30 minutes 
WWM1398: roger 


M 

BAY DEP: XXX, hold SW of CROIT on the SAU 035. right turns, climb and maintain 
10,000 expect further clearance in 30 minutes 


HOLDING MnEMMMLEI While in the holding pattern, subject will hear 


approach control issue long, extended delays to aircraft below him. 


SFO SPECIAL ATIS - READ. ID TEE NON -DAT A UNK CREWS 


21 

This is San Francisco airport information Echo 0215Z. Weather measured ceiling 300 overcast 
visibility 5. 'Temperature 50 dewpoint 41 wind light and variable altimeter 3004. The airport is 
temporarily closed due to a disabled aircraft on the runway. Estimated time to re-opening is unknown. 


HOLDING PATTERN EVENT #2: Subject overhears a VFR light aircraft call in at a 
nositiomjiea^ could be a confliction. 


N387R: Bay this is Cessna 387R just departed the Sacramento VOR headed toward 
Sausalito at 9,500 requesting advisories 

BAY DEP: Cessna 87R roger. be advised there is holding in progress on VI 50 from 10,000 
and below, squawk 0421 and ident 

BA Y DEP: DAL427 now expect further clearance in 1 hour 30 minutes, request your 
intentions 

DAL427: roger, stand by 

BAY DEP: Wings West 1398, now expect further clearance in 1 hour 30 minutes, the airport 
is still closed 

WWM1398: roger, we would like to go to Sacramento 
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ACES will probably call company and get routed 10 SMI: 

M 

COMPANY": Flight XXXX, be advised dispatch requests you divert to SMF. Contact 
dispatch with an estimate. 

After ACFS calls 

M 

BAY DEP: XXX turn right heading nnn. Intercept V150 to Sacramento maintain 10,000 

SIMILAR CALL SIGN EVENT; Subject is told of a civil jet on the frequency 
with a like sounding number. 

BAY DEP: XXX be advised there is an American 727 on the frequency with the same 
number as you 

BAY DEP: AALnnnn climb and maintain 1 1,000, contact Oakland Center on 124.2 
AALnnnn: cleared to 1 1,000 and changing 

aLH 

BAY DEP: XXX contact Oakland Center on 124.2 

After ACFS Calls 

OAK CENTER: XXX Oakland Center at 10,000 roger 

when ACFS is abeam CCR 

M 

OAK CENTER: XXX descend and maintain 7,000 contact Sacramento Approach on 125.6 
ATIS ( SMF ARRIVAL) 

ia 

This is Sacramento Metropolitan airport information Alpha 0245Z. Weather clear visibiluy50 
Temperature 58 dewpoint 39 wind 340 at 15 altimeter 3004. Runways 34 in use. Advise on initial 
contact you have Alpha. 

When ACFS is 10-15 SW of SAC VQR 


i! 

SAC APP: XXX roger fly heading nnn and intercept the 34 left localizer, descend and 
maintain 5,000, be advised there is a civil jet on the frequency with a similar number 

M 

SAC APP: XXX descend and maintain 3,000 

when ACES is 10 miles from airport 
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M 

SAC APP: XXX n miles from LANEE. cleared for ILS 34 left approach, contact the tower 
at LANEE 

After ACFS Calls 

M 

SMF TWR: XXX roger number 2 following a C5 eight miles ahead, caution wake turbulence 

AMWAY32: AMWAY32 on the touch and go requesting another approach 

SMF TWR: AMWAY32 roger, maintain runway heading, contact approach on 125.6 

<1 % 

SMF TWR: XXX traffic 1 1 o’clock 4 miles southeast bound altitude unknown 


M 

SMF TWR; XXX wind 340 at 15, cleared to land 34 left 

on rollout 

M 

SMF TWR: XXX turn right first available taxiway, contact ground on 1 21.7 clearing 

After ACFS Calls 

ML 

SMF GND: XXX roger taxi to gate 


Sacramento to San Francisco: 
ATIS (SMF DEPARTURE) 


This is the Sacramento Metropolitan airport information Charlie 0045Z. Weather 2500 scattered 
visibility 10. Temperature 55 dewpoint 42 wind 160 at 10 altimeter 2996. Runways 16 in use. Advise 
on initial contact you have Charlie. 

CLIMB VTA RUNWAY HEADING FOR RADAR VECTORS TO SACRAMENTO VOR, 
RISTI 2 ARRIVAL TO RUNWAYS 28 AT SAN FRANCISCO. 

After ACFS Calls 


SMF CLR: Cleared to San Francisco via runway heading vectors to SAC, RIS'l'I 2 arrival, 
maintain 5,000 expect 11,000 three minutes after departure. Departure control frequency will 
be 125.2 squawk 4331 


After ACFS Calls 
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II 

SMF GND: XXX taxi to runway 1 6 right via the parallel 
N17X: Ground, this is Gulfstream 17X for taxi 
SMF GND: Gulfstream! 7X taxi to runway 16R via the parrallel 
N17X: 17X roger 

N17X: Tower this is Gulfstream 17X ready for takeoff 
SMF TWR: Gulfstream 17X wind 170 at 10 cleared for takeoff 
N17X: 17X's rolling 

After ACFS Calls 

II 

SMF TWR: XXX taxi into position and hold 

SMF TWR: XXX wind 160 at 10, cleared for takeoff 
SMF TWR: 17X contact departure 
N17X: 17X roger good day 

when ACFS leaves 500’ 

M 

SMF TWR: XXX contact departure on 125.2 

After ACFS Calls 

SAC DEP: XXX this is Sacramento departure radar contact, ammend your clearance cleared 
direct Manteca, then the RISTI 2 arrival, maintain 5,000 

when ACFS is about 10 south of airport 

M 

SAC DEP: XXX traffic 2 o'clock 10 miles eastbound unverified altitude 5,500 

M 

SAC DEP: XXX Climb and maintain 9,000 contact Oakland center on 124.2 


TRAFFIC EVENT: T raffic is exchanged between ACFS and VFR conflict. The second 
time only tire VFR is told about traffic 


After ACFS Calls 
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it 

OAK CTR: XXX Oakland center roger climb and maintain 1 1,000 

u 

OAK CTR: XXX, Traffic 1 1 o'clock, 12 miles northwest bound, VFR at 9,500 
OAK CTR: N42X, traffic 1 o'clock 1 1 miles southeast bound, a United jet at 9,000 
N42X: roger were looking 

OAK CTR: N42X previous traffic now a 2 o'clock 5 miles southeast bound 
N42X: roger were still looking 
ATIS (SFO ARRIVAL) 

This is San Francisco airport information Golf 0045Z. Weather 3000 scattered visibility 50. 
Temperature 54 dewpoint 41 wind 210 at 5 altimeter 2996. Traffic landing runways 28 departing 
runways 1. Advise on initial 
contact you have Golf. 

when ACFS is approaching CEDES 
Il.3. 

OAK CTR: XXX contact Bay approach on 134.5 

IWRONG information j^Vj^Vl Subject is told he is on a vector for the wrong runway. | 

After ACFS Calls 

BA Y AFP: DAL29S fly heading 280 for vectors to 29R 
DAL298: Right to 280 for 29R 

M 

BAY APP: XXX fly heading 280 for vectors to 29R 

After ACFS sees the error put them on a heading for CEDES 

BAY APP: XXX bay approach, cross CEDES at 1 1,000 maintain 7,000 depart CEDES 
heading 240 and intercept the 28 right localizer 

[ SIMILAR CALL SIGN EVENT; Subject is told of a civil jet on the frequency I 
I with a like sounding number. I 

BAY APP: XXX be advised there is a company aircraft on the frequency with a similar 
number 

BA Y APP: UAL105 descend and maintain 6,000 
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UAL 105: roger, out of 10,000 for 6,000 

BAY APP: UAL105 reduce to 200 knots, contact Bay on 121.3 

UAL 105: reducing and going to 121.3 


when ACFS is abeam San Jose 

M 

BAY APP: XXX descend and maintain 6,000, contact the Bay on 135.6 

After ACFS Calls 

£2 

BAY APP: XXX roger, descend and maintain 4,000, reduce to 180 knots 

when ACFS is 15 miles from airport 

M 

BAY APP: XXX 10 miles from BRIJ.T, cleared for lUS 28 right, reduce to and maintain 160 
knots to BRIJJ contact tower at 10 DME. 

After ACFS Calls 


SFO TWR: XXX roger. wind 210 at 10 cleared to land 28R. 

on rollout 


SFO TWR: XXX turn left first available taxiway, cross 28 left contact ground 1 21.8 


clearing 


After ACFS Calls 

21 


SFO GND: XXX roger, taxi to gate 


San Francisco to Los Angeles: 

ATIS (SFO DEPARTURE) 

11 

This is San Francisco airport information Hotel 0145Z. Weather 3000 scattered visibility 50. 
Temperature 55 dewpoint 41 wind 200 at 5 altimeter 2996. Traffic landing and departing runways 2S. 
Advise on initial contact you have Hotel. 

PORTE 8 DEPARTURE, A VENAL TRANSITION, DIRECT FILMORE FILMORE 6 
ARRIVAL (RUNWAY 24/25 PROFILE DESCENT) TO RUNWAYS 24/25 AT LOS 
ANGELES 
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After ACFS Calls 


SFQ CLR: XXX Cleared as Sled via the Porte 8, maintain 1 1,000, expect flight level 290 10 
minutes after departure. Departure control frequency will be 120.9 squawk 4115 

After ACFS Calls 

1A 

SFO GND: XXX taxi runway 28 right via the outer and foxtrot hold short of the 1 's 

m 

SFO GND: XXX cross runways one contact tower on 120.5 

11 

SFO TWR: XXX Cross 28 left, taxi into position and hold 28 right 

After ACFS Calls 

21 

SFO TWR: XXX wind 210 at 5, cleared for takeoff 

after ACFS passes 500' 

21 

SFO TWR: XXX contact departure on 120.9 

After ACFS Calls 

u 

BAY DF.P: XXX Bay departure, radar contact 
BAY DEP: United 689 contact Oakland center on 128.7 

H 

BAY DEP: XXX fly heading 180 vector for spacing 

AS A3 11: Bay tins is ASA311 off 28 right climbing to 1 1,000 

BAY DEP: Alaska 31 1 Bay departure radar contact 

11 

BAY DEP: XXX turn left heading 060, intercept and resume the PORTE 8 departure, climb 
and maintain flight level 230, contact Oakland center on 128.7 

After ACFS Calls 

H 

OAK CTR: XXX expect to cross Pesca at or above 16,000 
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Before ACTS reaches 1 4,00.0 




: m 


M 

OAK CTR: XXX Oakland center cross Pesca at or above 16,000 expedite through 18.000 
traffic 1 o'clock 5 miles westbound altitude unknown 

after ACTS passes 20,000 

OAK CTR: UAL6S9 contact the center on 134.5 

M 

OAK CTR: XXX contact die center on 134.5 

After ACFS Calls 

M 

OAK CTR XXX Climb and maintain flight level 290 

M 

OAK CTR: XXX traffic 10 o'clock 12 miles westbound below- you 
ASA31 1 : Center Alaska 31 1 climbing out of nnn for flight level 230 
OAK CTR: Alaska .31 1 roger, climb and maintain flight level 290 

OAK CTR: United 689 ammend clearance, fly heading 140 receiving San Marcus proceed 
direct then via the SADDE 4 arrival to Los Angeles, maintain flight level 290 

UAL689: United 689 roger direct San Marcus and the SADDE 4 arrival 



OAK CTR: XXX, ammend clearance, fly heading 130 receiving San Marcus proceed direct 
then via the SADDE 4 arrival to Los Angeles, maintain flight level 290 

OAK CTR: United 689 contact Los Angeles center on 135.3 

When ACFS passes abeam Avenal 

M 

OAK CTR: XXX contact Los Angeles center on 135.3 

A Tier ACES (Tills 

LAX CTR : XXX Los Angeles center, roger 
ASA311: Center, Alaska 31 1 here at 310 
LAX CTR: Alaska 3 1 1 Los Angeles center roger 
L AX CTR: United 689 contact the center on 132.6 
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UAL689: United 689 roger 


M 

LAX CTR: XXX contact the center on 132.6 

After ACFS Calls 


M 

LAX CTR : XXX cleared to LAX via the SADDF. 4 arrival cross FIM at or below 190 
maintain 1 50. Los Angeles altimeter 30. 10 

ASA311: Los Angeles Alaska 311 out of flight level 310 on the SADDE 4 arrival 
LAX CTR: Alaska 3 1 1 roger Los Angeles altimeter nn.nn 

U 

Tliis is the Los Angeles airport information Yankee 0345Z. Weather clear visibility 50. Temperature 53 
dewpoint 30 wind 310 at 20, peak gusts 25 altimeter 2993. Traffic landing and departing runways 24 
and 25. Advise on initial contact you have Yankee. 


M 

LAX CTR: XXX contact Los Angeles Approach on 135.2 ((his &JL tiiSL for th£JZ£KJ£ 
respond & a wrons frequency. If the crew s hr/uld become loti ill tiiL frcwci-’Q i SL>d Qlllini , 
use the (ojlmdnz 


LAX CTR: Advisory: Communications link failure, contact Los Angeles center on 132.6 
ATIS (LAX ARRIVAL) 

LAX CTR: United 689 contact Los Angeles approach on 124.5 
UAL689: roger changing 

M 

LAX CTR: XXX reduce speed to 280 knots descend to cross Symcn at 12000. 

M 

LAX CTR: XXX contact Los Angeles approach on 124.5 

After ACFS Calls 

LAX APP: XXX Los Angeles approach roger 

ASA31 1 : Approach, Alaska 31 1 out of nnn for nnn 

LAX APP: Alaska 3 1 1 roger let me know when you have information xray 

ASA311: roger, we have xray 
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M 

LAX APP: XXX cross Bayst at 10000 

SEOI'ENCTNC F.VFNT : Subject has been following and is being followed by 
the same aircraft for some time. Subject (and traffic) is vectored downwind north 
of LAX in a normal way. Subject hears lead aircraft, and subsequently the 
following aircraft turned to base leg and subject is not. 


2 1 

LAX APP XXX Depart SMO heading 070 descend and maintain 6000. 

LAX APP: UAL689 turn right heading 160, descend and maintain 4,000 
UAL689: roger right to 160 and we're out of nnn for 4.000 

LAX APP: Alaska 311 tum right heading 160 for base leg, descend and maintain 4,000 
ASA311: roger out of nnn for 4.000 heading 160 

ACFS may ask for tum to base 


M 

LAX APP: XXX roger, vrc had to vector you a Hide farther downwind for spacing, turn 
right heading 160 for base leg, descend and maintain 4.000 

LAX APP: UAL689 nn miles from I.JMMA turn right heading 210 maintain nn until 
established on the localizer, cleared for ILS 25 right approach contact tower at LIMMA 

UAL689: Cleared for approach 

LAX APP: AS A3 1 1 nn miles from LIMMA turn right heading 210 maintain nn until 
established on the localizer, cleared for ILS 25 right approach contact tower at LIMMA 


ASA31 1: roger 


LAX APP; XXX nn miles from LIMMA turn right heading 210 maintain 3,500 until 
established on the localizer, cleared for ILS 25 right approach contact tower on 133.9 at 
LIMMA 


After ACTS Calls 

IM 

LAX TWR: XXX Los Angeles tower roger wind 310 at 20 cleared to land 25 right 

WIND SHI AH EVENT: The ACFS will experience an airspeed loss of 15 knots on 
final. They may elect to go around. 

UAL689: Ah tower we experienced a 15 knot loss at 500 ft., you might want to warn othe 
aircraft 
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LAX TWR: Roger, ASA311 did you copy that , the B727 ahead of you had a 15 knot loss 
on final 


ASA311: This is ASA31 1 roger, we just experienced the loss, were going around 

LAX TWR: ASA31 1 roger, maintain runway heading, climb and maintain 3,000, contact 
approach on 124.5 


ASA31 1: roger changing 


Ml 

LAX TWR: XXX be advised that two aircraft ahead of you experienced a fifteen knot loss on 
final, wind 310 at 22, cleared to land 



LAX TWR: XXX maintain runway headine, climb and maintain 3,000, contact approach on 
124.5 


After ACFS Calls 

Ml 

LAX DEP: XXX Los Angeles departure roger turn right heading 070 climb and maintain 
4,000 

When ACFS is 10 to 15 miles on downwind 

1 IM 

LAX DEP: XXX turn right heading 160 descend and maintain 3.000 

When ACFS approaches final 
FOR 25 RIGHT 


MI 

LAX DEP: XXX turn right heading 210, 7 miles form L1MMA, maintain 3,000 until 
established on the localizer, cleared for ILS 25 right approach, contact the tower on 133.9 at 
LIMMA 


After ACFS Calls 


Ml 

L.AX TWR: XXX Los Angeles tower roger wind 310 at 20 cleared to land 25 right 

on rollout 



Ml 


LAX DEP: XXX turn right heading 210, 7 miles form ROMEN, maintain 3,000 until 
established on the localizer, cleared for ILS 24 right approach, contact the tower on 133.9 at 
ROMEN 


After ACFS Calls 


LAX TWR: XXX Los Angeles tower roger wind 310 at 20 cleared to land 24 right 


LAX TWR: XXX turn right first available, contact ground on 121 .7 clearing 


311 ID 

LAX TWR: XXX turn left first available, contact ground on 121.7 clearing 

After ACFS Calls 


111 

LAX GND: XXX roger taxi to gate 
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Appendix F: 

Simulation PLI Event Analysis 
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Simulation PLI Event Analysis 


For each crew and event a set of questions were invoked which paralleled the 
scoring criteria from 1 to 5 described in Chapter 4. The answers to each question was used 
as a guideline for determining the level of crew response to the PLI present from which the 
score for that event was derived. 


PLI Aircraft holding short at taxiway intersection if "yes” then minimum 

score is at least 

a. No indication of awareness. 1 

b. Does the crew see or look for the other aircraft? 2-3 

c. Is crew concerned that aircraft will hold short for them? 3-4 

d. Does the crew query ATC concerning the traffic? 5 

e. Does the crew stop the simulator? 5 


The actual scores for all crew responses to PLI based on this set of questions are summarized as follows. 

Crew # 1 2 2 4 5 6 2 

Score 2 2 5 5 1 2 3 


These scores transfer to the summary of results in Table 4.2 as shown below. 


Not Action 

Aware Aware Taken 

1 4 2 


PL2 Aircraft crossing active runway 

a. No indication of awareness. 

b. Does the crew hear about the other a/c? 

c. Does the crew know he is crossing in front of them? 

d. Does the crew query ATC about the traffic? 

e. Does the crew abort the takeoff? 


if "yes" then minimum 
score is at least 
1 

2-3 

34 

5 

5 


The actual scores for all crew responses to PL2 based on this set of questions are summarized as follows. 

Crew # 1 2 2 i 5 6 1 

Score 1 1 1 1 1 E 1 


These scores transfer to the summary of results in Table 4.2 as shown below. 


Not 

Aware 

6 


Action 

Ta k en 

0 


E = script error 


Aware 

0 
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PL3 Turbulence and Weather Deviations if "yes" then minimum 

sc ore i s at i c a&i 

a. No indication of awareness. 1 

b. Does the crew hear the other a/c report turbulence? 2 

c. Does the crew know he is slowing? 3 

d. Is the crew concerned about going thru turbulence? 4 

e. Does the crew request a course or altitude deviation? 5 


The actual scores for all crew responses to PL3 based on this set of questions are summarized as follows. 

Crew.# 1 2 2 4 1 

Score 5 5 5 5 1 4 5 


These scores transfer to the summary of results in Table 4.2 as shown below. 

Not Action 

Aware Aware Taken 

1 1 5 


PL4 Aircraft on Runway of Intended Landing if "yes” then minimum 

s core is a t l ea st 

a. No indication of awareness. I 

b. Does the crew know the other a/c is on runway? 2-3 

c. Does the crew state that they may have to go around? 34 

d. Does the crew initiate a go around with ATC instructions? 4-5 

e. Does the crew initiate a go around without ATC instructns? 5 


The actual scores for all crew responses to PL4 based on this set of questions are summarized as follows. 

Crew # 1 2 2 4 5 6 1 

Score 4 2 4 4 4 4 4 


These scores transfer to the summary of results in Table 4.2 as shown below. 

Not Action 

Aware Aware Taken 

0 7 0 


PL5 Holding EFC Validity if "yes” then minimum 

score is at least 

a. No indication of awareness. I 

b. Does the crew indicate hearing the other a/c's EFC? 2-3 

c. Does this cause the crew to discuss diverting? 4 

d. Does the crew decide to divert before the company calls? 5 
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The actual scores for all crew responses to PL5 based on this set of questions are summarized as follows. 

Crew # 1 2 3 4 5 6 1 

Score 5 5 5 5 5 2 5 


These scores transfer to the summary of results in Table 4.2 as shown below. 

Not Action 

Aware Aware Taken 

0 1 6 


PL6 Traffic watch in hold if "yes" then minimum 

score is at least 

a. No indication of awareness. 1 

b. Does the crew indicate hearing other aircraft? 2-3 

c. Does this cause them to look for the traffic? 3-4 

d. Does the crew discuss a possible conflict? 4 

e. Does the crew query ATC about the traffic? 5 

f. Does the crew modify the simulator flight path? 5 

The actual scores for all crew responses to PL6 based on this set of questions are summarized as follows. 

Crew # 1 2 2 4 5 6 1 

Score 113 4 111 

These scores transfer to the summary of results in Table 4.2 as shown below. 

Not Action 

Aware Aware Taken 

5 2 0 


PL7 Traffic Watch During Climb if "yes” then minimum 

score is at least 

a. No indication of awareness. 1 

b. Does the crew hear traffic advisory given to other a/c? 2-3 

c. Does crew see the traffic? 34 

e. Does the crew modify the simulator flight path? 5 


The actual scores for all crew responses to PL7 based on this set of questions are summarized as follows. 

Crew # 1 2 2 4 5 £ 1 

Score 2 2 4 3 2 2 1 


These scores transfer to the summary of results in Table 4.2 as shown below. 


Not 

Mass. 

l 


Action 

Taken 

0 


Aware 

6 
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PL8 Aircraft Sequencing 


a. No indication of awareness. 

b. Does the crew indicate knowing they are in a sequence? 

c. Does the crew know the a/c ahead of them gets turned? 

d. Does the crew know the a/c behind them gets turned? 

e. Does the crew query ATC or request a turn? 


if "yes" then minimum 
score is at least 

1 

2- 3 

3- 4 
34 
5 


The actual scores for all crew responses to PL8 based on this set of questions are summarized as follows. 

Crew # 1 2 2 4 5 6 2 

Score 2 2 2 M 1 3 2 

M = missing data 

These scores transfer to the summary of results in Table 4.2 as shown below. 

Not Action 

Aware Aware Iak£Q 

1 5 0 


if "yes" then minimum 
score is at least 

1 

2-3 

4-5 

5 

5 


The actual scores for all crew responses to PL9 based on this set of questions are summarized as follows. 

Crew # 1 2 2 4 5 fi 1 

Score 5 5 5 M 5 5 5 


PL9 Windshear on Final Approach 

a. No indication of awareness. 

b. Does the crew hear other a/c report airspeed loss? 

c. Does the crew query ATC for more info? 

d. Does the crew adjust their approach speed? 

e. Does the crew go around? 


m 



M = missing data 

These scores transfer to the summary of results in Table 4.2 as shown below. 

Not Action 

Aware Aware Taken 

0 0 5 




tl 

f i 


fcJ 


, : ; 
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